dohertj2/lmxopcua
1
0
Fork 0
Code Issues 14 Pull Requests Actions Packages Projects Releases Wiki Activity
Files
c74ccc55b5f3d9cfb3d32d89dff4370f9ad9f889
lmxopcua/docs/DriverLifecycle.md
T
Joseph Doherty 2d20aeb779 fix(opcua,drivers): carry driver status sub-codes to clients (#519) + health-write ordering (#517) 
#519 — a driver's OPC UA status code now reaches a client with its sub-code intact.
DriverInstanceActor projected the driver's uint onto a 3-state Good/Uncertain/Bad enum
and OtOpcUaNodeManager re-expanded Bad to a bare 0x80000000, so BadNoCommunication,
BadTypeMismatch and BadNotSupported were indistinguishable on the wire and every
driver's status constants (incl. the 16 #497 had just corrected) were diagnostic in
logs only. Resolved by option (b): the raw uint rides AttributeValuePublished ->
AttributeValueUpdate -> IOpcUaAddressSpaceSink.WriteValue, OpcUaQuality is deleted, and
severity classification is a helper (OpcUaStatus.IsGood/IsUncertain/IsBad).

The surface was wider than the issue costed (41 files, not "the sink and 3
implementations"): OpcUaQuality had non-driver producers with no uint to carry.
Each got its natural code — BadInternalError for a failed virtual-tag script,
BadNoCommunication for driver comms loss — and ScriptedAlarmHostActor's pair of
mutually-inverse uint<->enum mappers is deleted, having round-tripped a code the
engine already held. AddressSpaceApplier's removal writes deliberately keep the
generic Bad: not losing a code the driver had is this issue; inventing one for the
deploy path is a separate call.

Behaviour change: WriteAlarmQuality fires its Part 9 event on a status-code change
rather than a severity-bucket change. While the annotation was 3-state those were the
same predicate; now a move between two Bad reasons changes what a subscriber reads.

New SDK-free named constants live on Commons.OpcUa.OpcUaStatus, and
StatusCodeParityTests is widened to cover them — that guard is reflection-driven, so
it can only see a named constant, never an inline literal.

#517 — the symptom is real but the issue named the wrong path and its fix was unsafe.
ShutdownAsync cannot race: DriverInstanceActor calls it from PostStop, after the
mailbox drains. (Core/Hosting/DriverHost, the one class with a concurrent GetHealth,
has no production call site.) InitializeAsync can: it runs on a fire-and-forget
Task.Run, and MTConnectDriver writes Healthy then starts loops that can throw into a
catch which disposes the client before writing Faulted.

Moving the write earlier, as suggested, inverts the terminal state — teardown drains
in-flight loops that write health as they unwind, so a late loop could leave Healthy at
rest with every connection closed. The fix is a bracket (conservative before, terminal
after), applied to MTConnect's init-failure path only; shutdown keeps its single
trailing write, and the rule plus the reachability table are documented on
IDriver.ShutdownAsync.

CalculationDriver.GetHealth returned a hard-coded Healthy that consulted nothing —
Healthy before initialize, after shutdown and after dispose. Now derived from _running.
Its test, named Health_is_always_Connected, asserted the defect and is replaced.

Every new guard was falsified by breaking the production code and watching it go red.
The two sub-code tests each assert two different Bad codes, since a single-code
assertion could pass against a hard-coded constant under the old projection.

Pre-existing failures, verified unchanged on a stashed tree and not chased: 4 in
Driver.AbLegacy.IntegrationTests (slc500/micrologix/plc5 fixtures not up) and 3 in
Host.IntegrationTests.

Closes #517
Closes #519

Claude-Session: https://claude.ai/code/session_015p7wGqy3YpZNCpDzTpGMKo
2026-07-31 05:25:27 -04:00

330 lines
17 KiB
Markdown
Raw Blame History

# Driver Lifecycle & Server Infrastructure Contracts
Reference for the server-side infrastructure interfaces that surround a
driver but are **not** driver *capabilities* (read/write/subscribe/etc.,
documented in [ReadWriteOperations.md](ReadWriteOperations.md) and the
per-driver pages). These contracts live in
[`src/Core/ZB.MOM.WW.OtOpcUa.Core.Abstractions/`](../src/Core/ZB.MOM.WW.OtOpcUa.Core.Abstractions/)
so they carry no behavior — concrete implementations live in the driver
projects, the Runtime, and the ControlPlane. Each subsection below gives the
purpose, the key members, and where it is implemented/used.
The capability interfaces a driver opts into (`IReadable`, `IWritable`,
`ITagDiscovery`, `ISubscribable`, `IAlarmSource`, `IHistoryProvider`,
`IHostConnectivityProbe`, `IPerCallHostResolver`, `IRediscoverable`) are
covered elsewhere and discovered by the server via `is`-checks on the
`IDriver` instance. The interfaces here are the *plumbing* the server uses to
**create**, **probe**, **supervise**, **report on**, and **configure** those
drivers, plus the server-side historian read surface.
---
## IDriverFactory — creating drivers from config rows
[`Core.Abstractions/IDriverFactory.cs`](../src/Core/ZB.MOM.WW.OtOpcUa.Core.Abstractions/IDriverFactory.cs)
Abstraction over the process-wide driver registry. The Runtime consumes this
instead of the concrete registry so the Runtime project does not pull in
`ZB.MOM.WW.OtOpcUa.Core` (which would drag in Polly + driver hosting).
Members:
- `IDriver? TryCreate(string driverType, string driverInstanceId, string driverConfigJson)`
— returns a new driver for the given type, or `null` when no factory is
registered for that type (missing assembly, typo). The `DriverHostActor`
logs and skips the row rather than failing the whole apply.
- `IReadOnlyCollection<string> SupportedTypes` — driver-type names this
factory can materialise; mostly for diagnostics and logs.
Implementations:
- `NullDriverFactory` (same file) returns `null` from every `TryCreate` and
exposes zero supported types. Bound when no concrete driver assemblies have
been registered (Mac dev path, smoke tests); the deployment becomes a no-op.
- `DriverFactoryRegistry`
([`Core/Hosting/DriverFactoryRegistry.cs`](../src/Core/ZB.MOM.WW.OtOpcUa.Core/Hosting/DriverFactoryRegistry.cs))
is the real process-singleton registry keyed by `DriverInstance.DriverType`
(case-insensitive). Each driver project ships a `Register(...)` extension;
`Register` records the factory **and** the driver's stability
[`DriverTier`](../src/Core/ZB.MOM.WW.OtOpcUa.Core.Abstractions/DriverTier.cs)
(defaults to Tier A). Registering the same type twice throws.
- `DriverFactoryRegistryAdapter`
([`Core/Hosting/DriverFactoryRegistryAdapter.cs`](../src/Core/ZB.MOM.WW.OtOpcUa.Core/Hosting/DriverFactoryRegistryAdapter.cs))
bridges the registry to the `IDriverFactory` abstraction.
Wiring: `DriverFactoryBootstrap.AddOtOpcUaDriverFactories`
([`Host/Drivers/DriverFactoryBootstrap.cs`](../src/Server/ZB.MOM.WW.OtOpcUa.Host/Drivers/DriverFactoryBootstrap.cs))
registers the singleton registry, runs every driver assembly's `Register`
extension, then binds `IDriverFactory` to the adapter. It must run **before**
`AddAkka` so the Runtime can resolve `IDriverFactory` when spawning the
`DriverHostActor`
([`Runtime/Drivers/DriverHostActor.cs`](../src/Server/ZB.MOM.WW.OtOpcUa.Runtime/Drivers/DriverHostActor.cs)).
The registry is skipped on admin-only nodes (they never run drivers); the
probe set is the exception — see [IDriverProbe](#idriverprobe--test-connect).
---
## IDriverProbe — Test Connect
[`Core.Abstractions/IDriverProbe.cs`](../src/Core/ZB.MOM.WW.OtOpcUa.Core.Abstractions/IDriverProbe.cs)
A cheap test-connect probe for one driver type, backing the AdminUI **Test
Connect** button. An implementation deserializes a driver-config JSON, attempts
a cheap connection (TCP open, OPC UA session, gRPC ping — whatever the driver's
native protocol supports), and reports success/failure with latency. **Probes
must not mutate persistent state**: the AdminUI invokes them against the
transient config in the typed form, not against the persisted `DriverInstance`
row.
Members:
- `string DriverType { get; }` — the `DriverInstance.DriverType` string this
probe handles; used for DI lookup.
- `Task<DriverProbeResult> ProbeAsync(string configJson, TimeSpan timeout, CancellationToken ct)`
— never throws on connection failure; returns a result with `Ok = false`
and a message instead.
- `DriverProbeResult(bool Ok, string? Message, TimeSpan? Latency)` — outcome
record (`Message` is `null` on success; `Latency` is `null` on failure).
Implementations: every driver ships a `*DriverProbe` in its driver project
(e.g.
[`Driver.Modbus/ModbusDriverProbe.cs`](../src/Drivers/ZB.MOM.WW.OtOpcUa.Driver.Modbus/ModbusDriverProbe.cs)
does a bare socket open/close). The historian backend is the external
HistorianGateway (consumed as a gRPC client package, not an `IDriver`), so it
has no driver probe.
Flow: the AdminUI's `AdminProbeService`
([`AdminUI/Clients/AdminProbeService.cs`](../src/Server/ZB.MOM.WW.OtOpcUa.AdminUI/Clients/AdminProbeService.cs))
dispatches a `TestDriverConnect` message through `IAdminOperationsClient` to the
cluster-singleton `AdminOperationsActor`
([`ControlPlane/AdminOperations/AdminOperationsActor.cs`](../src/Server/ZB.MOM.WW.OtOpcUa.ControlPlane/AdminOperations/AdminOperationsActor.cs)),
which holds the probes keyed by `DriverType` and invokes the matching one
(timeout clamped to `[1, 60]` seconds). Because the admin singleton is
admin-pinned, the probe set must be registered on admin nodes too — `Program.cs`
calls `AddOtOpcUaDriverProbes` in the `hasAdmin` block, and
`AddOtOpcUaDriverFactories` registers it for fused admin+driver nodes.
---
## IDriverSupervisor / process recycle — REMOVED (Gitea #522)
`IDriverSupervisor`, `MemoryRecycle`, `MemoryTracking` and
`ScheduledRecycleScheduler` **no longer exist.** This section used to describe
them as the Tier C protection layer.
They were removed because the thing they acted on is gone. "Tier C" meant a
driver running **out-of-process behind a supervisor that could restart its Host**
without tearing down the OPC UA session or co-hosted drivers. No such process
remains anywhere: Galaxy reaches MXAccess over gRPC to the external
`mxaccessgw` sidecar (PR 7.2 retired the in-process `Galaxy.Host` / `Proxy` /
`Shared` projects), and FOCAS has run in-process since its managed wire client
landed 2026-04-24. Consistent with that, `IDriverSupervisor` had **zero
implementations**, and `MemoryTracking` / `MemoryRecycle` /
`ScheduledRecycleScheduler` were constructed **only in their own unit tests** —
so this was not dormant-but-ready code, it was code with no path to running.
The operator-facing half went too: `RecycleIntervalSeconds` was authorable
through the AdminUI's driver resilience section and validated by the parser
while configuring nothing. A stored `ResilienceConfig` still carrying the key
parses cleanly (unknown keys are ignored) and the AdminUI preserves it rather
than silently rewriting an operator's config.
**`DriverTier` itself survives and is load-bearing** — do not delete it while
following this note. `DriverResilienceOptions.GetTierDefaults(tier)` supplies
the real per-capability timeout / retry / breaker policies, resolved via
`DriverFactoryRegistry.GetTier` and applied by `DriverCapabilityInvokerFactory`.
Every driver runs Tier A because no factory passes a tier, so today that means
one uniform policy set.
⚠️ Two `IDriver` members are now **consumerless**: `GetMemoryFootprint()` and
`FlushOptionalCachesAsync()` existed to feed `MemoryTracking`. All 12 drivers
still implement them and nothing calls them. They were left in place rather
than swept, since removing them touches every driver and every test stub —
tracked as Gitea **#525**.
If per-driver memory protection is wanted again, build it for the architecture
as it is now: a process-wide watchdog, not a per-driver tier whose only remedy
requires a supervisor that cannot exist in-process.
---
## IDriverHealthPublisher — health pub/sub sink
[`Core.Abstractions/IDriverHealthPublisher.cs`](../src/Core/ZB.MOM.WW.OtOpcUa.Core.Abstractions/IDriverHealthPublisher.cs)
A sink for driver-health state-change notifications. Implementations must be
non-blocking and safe to call from any thread.
Member:
- `void Publish(string clusterId, string driverInstanceId, DriverHealth health, int errorCount5Min)`
Implementations:
- `NullDriverHealthPublisher` (same file) is the drop-in no-op for tests and
dev-stub paths. A `DriverInstanceActor` defaults to it when no publisher is
supplied.
- `AkkaDriverHealthPublisher`
([`Runtime/Drivers/AkkaDriverHealthPublisher.cs`](../src/Server/ZB.MOM.WW.OtOpcUa.Runtime/Drivers/AkkaDriverHealthPublisher.cs))
is the production binding: it forwards each transition as a
`DriverHealthChanged` message onto the cluster-wide `driver-health`
Akka DistributedPubSub topic.
Producer: `DriverInstanceActor`
([`Runtime/Drivers/DriverInstanceActor.cs`](../src/Server/ZB.MOM.WW.OtOpcUa.Runtime/Drivers/DriverInstanceActor.cs))
calls `Publish` when a driver's health transitions. The published snapshot is
consumed AdminUI-side and surfaced through the driver-status panel (read
in-process by the AdminUI bridge rather than dialing its own hub).
### Writing health around a teardown (Gitea #517)
Two rules, and the second is the counter-intuitive one:
1. **`GetHealth()` must report actual state, never a constant.** A hard-coded `Healthy` reports a
healthy driver before initialize and after shutdown alike. `CalculationDriver` did exactly this —
it was the fleet's one driver reporting Healthy with its timers stopped, deterministically rather
than as a race. A driver with no transport still has a running / not-running distinction.
2. **Land the terminal health value *after* the teardown — and, where the window is observable, write
it *before* as well.** #517's original suggestion was to move the write earlier so the race window
leans conservative. Moving it is wrong: teardown drains in-flight poll/probe loops, and those loops
write health as they unwind, so a driver that only wrote first could have a late loop overwrite the
shutdown state with `Healthy` and leave *that* as the resting value — with every connection closed.
The safe shape is a **bracket**: conservative value before, terminal value after.
Where the window is actually observable is worth knowing before adding brackets everywhere:
| Path | Caller | Can `GetHealth()` land mid-teardown? |
|---|---|---|
| `ShutdownAsync` | `DriverInstanceActor.PostStop` | **No** — actor thread, after the mailbox drains, so the health poll is serialised against it. |
| `InitializeAsync` failure | `DriverInstanceActor.InitializeAsync` | **Yes** — invoked on a fire-and-forget `Task.Run`, so the health-poll tick keeps calling `GetHealth()` throughout. |
So a driver whose `InitializeAsync` disposes anything on its failure path must bracket; only
`MTConnectDriver` does today, because it is the only one that tears down inside a `catch`. The
shutdown paths deliberately keep the single trailing write.
---
## IDriverConfigEditor — custom AdminUI config editor (plug-point)
[`Core.Abstractions/IDriverConfigEditor.cs`](../src/Core/ZB.MOM.WW.OtOpcUa.Core.Abstractions/IDriverConfigEditor.cs)
An **optional** plug-point a driver can implement to provide a custom AdminUI
editor for its `DriverConfig` JSON. Drivers that don't implement it fall back to
the generic JSON editor with schema-driven validation. This is the contract
between the driver and the Admin Blazor app; the Admin app discovers
implementations and slots them into the Driver Detail screen.
Members:
- `string DriverType { get; }` — the driver type this editor handles.
- `Type EditorComponentType { get; }` — the Razor component type that renders
the editor (returned as `Type` so `Core.Abstractions` needs no Blazor
reference).
Status: this is a forward-looking plug-point. No driver ships a concrete
`IDriverConfigEditor` today — every driver uses the generic JSON editor — so
the interface currently has the contract defined but no implementations.
---
## IHistorianDataSource — server-side historian read surface
[`Core.Abstractions/Historian/IHistorianDataSource.cs`](../src/Core/ZB.MOM.WW.OtOpcUa.Core.Abstractions/Historian/IHistorianDataSource.cs)
The server-side historian read surface. Registered with the server's history
router and resolved **per OPC UA namespace**, independent of any driver's
lifecycle. This is distinct from the driver capability `IHistoryProvider`:
- `IHistoryProvider` is a *driver capability* — the server dispatches to it via
the driver instance.
- `IHistorianDataSource` is a *server registration* — the server resolves it by
namespace and calls it directly, so one historian (the HistorianGateway) can
serve many drivers' nodes, and drivers can restart without dropping history
availability.
The interface is `: IDisposable` and declares the full read surface as
**required** members (unlike `IHistoryProvider`, where at-time/event reads are
optional default-impl methods so legacy drivers can stay raw-only):
- `ReadRawAsync(fullReference, startUtc, endUtc, maxValuesPerNode, ct)` — raw
historical samples over a time range.
- `ReadProcessedAsync(fullReference, startUtc, endUtc, interval, aggregate, ct)`
— interval-bucketed aggregates (average/min/max/count); an empty bucket
returns a `BadNoData` sample.
- `ReadAtTimeAsync(fullReference, timestampsUtc, ct)` — one sample per requested
timestamp (OPC UA HistoryReadAtTime); the returned list matches the requested
length and order, gaps as Bad-quality snapshots.
- `ReadEventsAsync(sourceName, startUtc, endUtc, maxEvents, ct)` — historical
alarm/event records (OPC UA HistoryReadEvents); `sourceName` is `null` to
return all sources. `maxEvents` is a signed `int` so a non-positive value is a
"use the backend's default cap" sentinel.
- `GetHealthSnapshot()` — point-in-time health snapshot for diagnostics and
dashboards; pure observation, never blocks on backend I/O.
All values use the shared `DataValueSnapshot` / `HistoricalEvent` shapes;
backend-specific quality/type encodings are translated to OPC UA `StatusCode`
uints inside the data source.
Implementation:
- `GatewayHistorianDataSource`
([`Driver.Historian.Gateway/GatewayHistorianDataSource.cs`](../src/Drivers/ZB.MOM.WW.OtOpcUa.Driver.Historian.Gateway/GatewayHistorianDataSource.cs))
— the read backend that talks gRPC to the external `ZB.MOM.WW.HistorianGateway`
(via the `ZB.MOM.WW.HistorianGateway.Client` package, behind the
`IHistorianGatewayClient` seam). The alarm-event drain target is the separate
`GatewayAlarmHistorianWriter` (the gateway `SendEvent` path; see
[AlarmHistorian.md](AlarmHistorian.md)).
The HistorianGateway is the sole historian backend; its config keys and
deployment prerequisites are in [Historian.md](Historian.md).
---
## Commons — shared cross-cutting primitives
[`src/Core/ZB.MOM.WW.OtOpcUa.Commons/`](../src/Core/ZB.MOM.WW.OtOpcUa.Commons/)
`ZB.MOM.WW.OtOpcUa.Commons` is the low-level shared library that the Runtime,
ControlPlane, AdminUI, and OPC UA server projects all reference. It holds
cross-cutting primitives with no driver- or host-specific behavior, so the
heavier projects can share message contracts and value types without taking a
dependency on each other. It references only `Akka` and the internal
`ZB.MOM.WW.Audit` package.
Folders:
- **`Messages/`** — Akka message contracts grouped by concern (`Admin`,
`Alerts`, `Deploy`, `Drivers`, `Fleet`, `Logging`, `Redundancy`). These are
the wire/inter-actor messages — e.g. `Messages/Admin/TestDriverConnect.cs`
(Test Connect request, see [IDriverProbe](#idriverprobe--test-connect)) and
`Messages/Drivers/DriverHealthChanged.cs` (the driver-health pub/sub payload,
see [IDriverHealthPublisher](#idriverhealthpublisher--health-pubsub-sink)).
- **`Interfaces/`** — cluster-facing client contracts such as
`IAdminOperationsClient`, `IClusterRoleInfo`, and `IFleetDiagnosticsClient`.
- **`Types/`** — strongly-typed identifier value types: `CorrelationId`,
`DeploymentId`, `ExecutionId`, `NodeId`, `RevisionHash`.
- **`Browsing/`** — live-browse abstractions (`BrowseNode`, `IBrowseSession`,
`IDriverBrowser`) backing the AdminUI address pickers.
- **`Engines/`** — evaluator seams (`IScriptedAlarmEvaluator`,
`IVirtualTagEvaluator`, `IAlarmActorStateStore`) consumed by the
[VirtualTags](VirtualTags.md) / [ScriptedAlarms](ScriptedAlarms.md) engines.
- **`OpcUa/`** — deferred-publish seams (`IOpcUaAddressSpaceSink`,
`IServiceLevelPublisher` and their `Deferred*` no-op stand-ins) so address-space
and [ServiceLevel](Redundancy.md) writes can be wired late.
- **`Observability/`** — `OtOpcUaTelemetry` (the shared ActivitySource/metrics
surface).
---
## See also
- [ReadWriteOperations.md](ReadWriteOperations.md) — the driver *capability*
interfaces (read/write/subscribe) and resilience pipeline.
- [ServiceHosting.md](ServiceHosting.md) — role gating, the Akka cluster, and
the external HistorianGateway backend.
- [AlarmHistorian.md](AlarmHistorian.md) — the store-and-forward SQLite alarm
sink that drains to `IAlarmHistorianWriter`.
- [Redundancy.md](Redundancy.md) — driver stability tiers in the redundancy
context.
Reference in New Issue View Git Blame Copy Permalink