feat(mtconnect): CommsLossConfirmMs AdminUI field, refused-connection fast path reaches native conditions, dedicated live-gate Agent (#852)
- AdminUI MTConnect driver form: "Comms-loss confirm (ms)" in the Reconnect section, always written (0 = first refusal, negative = off is never dropped), case-insensitive load, unknown keys preserved, advisory at >= 90000 ms (the watchdog fires first). Form output byte-pinned and parsed through MTConnectDriver.ParseOptions on the driver side. - Driver: a confirmed refused-connection outage now annotates every told CONDITION quality-only BadNoCommunication (state/ack kept, once per outage, UNAVAILABLE-Bad skipped); the first delivery afterwards re-diffs all subscribed conditions so unchanged ones get a quality-only Good. - docker-dev/docker-compose.mtconnect-gate.yml: throwaway Agent in its adapter's network namespace so a stop is a genuine ConnectionRefused (a plain stopped container is a DNS failure = ambiguous). - Docs: MTConnect.md comms-loss section + live-gate recipe, docker-dev README, drivers.md rule. Refs #852
This commit is contained in:
Joseph Doherty
@@ -317,6 +317,12 @@ host logger factory (wired in `DriverFactoryBootstrap`; the FOCAS and TwinCAT fa
|
||||
every subscribed value `BadNoCommunication` once per outage; the first delivery afterwards republishes all of them from
|
||||
the index. Health/`LastSuccessfulRead` are untouched by it (the stream failure already degraded), so the #527 watchdog
|
||||
still re-inits at 90 s. Timeouts/resets/DNS/pause stay watchdog-only — never widen the definite set to them.
|
||||
**#852:** the same moment annotates every TOLD native condition quality-only `BadNoCommunication` (state/ack kept,
|
||||
recorded in the told record; one already UNAVAILABLE-Bad is skipped), and recovery re-diffs ALL subscribed conditions
|
||||
(`EmitConditionTransitions(null, resync: false)`) so unchanged ones get a quality-only Good. AdminUI field "Comms-loss
|
||||
confirm (ms)" is ALWAYS written (a negative = off must never be dropped like a non-positive timing knob — that would
|
||||
re-enable the 2000 default). Live gate: a plainly `docker stop`ped Agent drops out of Docker DNS ⇒ HostNotFound
|
||||
(ambiguous) — use `docker-dev/docker-compose.mtconnect-gate.yml` (Agent in the adapter's netns ⇒ real refusal).
|
||||
- **MTConnect CONDITION alarms (#626):** a CONDITION raw tag is a native Part 9 condition only when its `TagConfig`
|
||||
carries the standard `alarm` object (opt-in — alarm tags leave the value subscription, so auto-promotion would break
|
||||
existing String tags). Normal→inactive, Warning→500, Fault→800; one condition per DataItem showing the most severe
|
||||
|
||||
@@ -198,6 +198,25 @@ dotnet test tests/Drivers/ZB.MOM.WW.OtOpcUa.Driver.Historian.QuestDb.Integration
|
||||
Tear down with `docker compose -f docker-dev/docker-compose.yml -f docker-dev/docker-compose.questdb.yml rm -sf questdb` (add `docker volume rm otopcua-questdb-data`
|
||||
to drop the data). The `admin`/`quest` credentials are QuestDB's stock defaults, not secrets.
|
||||
|
||||
## MTConnect refused-connection gate overlay (#852)
|
||||
|
||||
`docker-compose.mtconnect-gate.yml` is an **optional overlay** that adds a dedicated, throwaway MTConnect Agent
|
||||
(`mtconnect-gate-agent`, `mtconnect/agent:2.7.0.12`) plus its SHDR adapter (`mtconnect-gate`), reusing the integration
|
||||
fixture's `agent.cfg` / `Devices.xml` / `adapter.py` — so the #846/#852 fast path can be gated by stopping an Agent
|
||||
**without** touching the shared fixture on `10.100.0.35`. Driver URI from inside the rig: `http://mtconnect-gate:5000`;
|
||||
from the Mac: `http://127.0.0.1:5155`. The Agent runs in the adapter's network namespace so that stopping it yields a
|
||||
genuine connection **refusal** (a plainly stopped container drops out of Docker DNS — an ambiguous DNS failure the fast
|
||||
path deliberately ignores).
|
||||
|
||||
```bash
|
||||
docker compose -f docker-dev/docker-compose.yml -f docker-dev/docker-compose.mtconnect-gate.yml up -d --no-deps mtconnect-gate mtconnect-gate-agent
|
||||
docker compose -f docker-dev/docker-compose.yml -f docker-dev/docker-compose.mtconnect-gate.yml stop mtconnect-gate-agent # refuse
|
||||
docker compose -f docker-dev/docker-compose.yml -f docker-dev/docker-compose.mtconnect-gate.yml start mtconnect-gate-agent # recover
|
||||
docker compose -f docker-dev/docker-compose.yml -f docker-dev/docker-compose.mtconnect-gate.yml rm -sf mtconnect-gate-agent mtconnect-gate
|
||||
```
|
||||
|
||||
Full gate recipe (driver config, Client CLI, expectations): [`docs/drivers/MTConnect.md`](../docs/drivers/MTConnect.md#live-gate-refused-connection-fast-path-846852--recipe).
|
||||
|
||||
## Tear down
|
||||
|
||||
```bash
|
||||
|
||||
@@ -0,0 +1,66 @@
|
||||
# Dedicated, THROWAWAY MTConnect Agent for the #846/#852 refused-connection live gate.
|
||||
#
|
||||
# Adds two services to the docker-dev rig's own network so a driver deployed on
|
||||
# central-1/central-2 can reach an Agent that the gate is free to stop and start —
|
||||
# never the shared fixture on 10.100.0.35, which other work relies on. It reuses the
|
||||
# integration fixture's agent.cfg / Devices.xml / adapter.py (read-only binds), so
|
||||
# the device model is the familiar `OtFixtureCnc` one (fixture_x_pos, fixture_execution,
|
||||
# the fixture_x_travel / fixture_logic CONDITIONs...).
|
||||
#
|
||||
# WHY THE AGENT SHARES THE ADAPTER'S NETWORK NAMESPACE (the whole point of this file):
|
||||
# the fast path fires ONLY on SocketError.ConnectionRefused. Stopping an ordinary
|
||||
# compose container also drops its name from Docker's embedded DNS, so the driver
|
||||
# would see a DNS failure (HostNotFound) instead — an AMBIGUOUS failure that, by
|
||||
# design, waits for the ~90 s watchdog, and the gate would "fail" for the wrong
|
||||
# reason. Here the name the driver dials (`mtconnect-gate`) belongs to the ADAPTER
|
||||
# container, which keeps running; the Agent joins its namespace
|
||||
# (network_mode: service:...), so stopping the Agent leaves the name resolving to an
|
||||
# IP with nothing on :5000 — the kernel answers RST, i.e. a genuine refusal.
|
||||
# (Stopping BOTH services is the ambiguous control leg: DNS failure, watchdog only.)
|
||||
#
|
||||
# up: docker compose -p otopcua-dev -f docker-dev/docker-compose.yml \
|
||||
# -f docker-dev/docker-compose.mtconnect-gate.yml up -d mtconnect-gate mtconnect-gate-agent
|
||||
# refuse: docker compose -p otopcua-dev -f docker-dev/docker-compose.yml \
|
||||
# -f docker-dev/docker-compose.mtconnect-gate.yml stop mtconnect-gate-agent
|
||||
# recover: ... start mtconnect-gate-agent
|
||||
# remove: ... rm -sf mtconnect-gate-agent mtconnect-gate
|
||||
#
|
||||
# Driver AgentUri (from inside the rig) -> http://mtconnect-gate:5000
|
||||
# From the Mac (curl /probe) -> http://127.0.0.1:5155 (macOS squats :5000 — AirPlay)
|
||||
#
|
||||
# Starting only these two services touches nothing else in the rig (no depends_on
|
||||
# either way). Recipe: docs/drivers/MTConnect.md §"Live gate: refused-connection fast path".
|
||||
services:
|
||||
|
||||
# Network-namespace owner + SHDR data source. Must be started first and must stay
|
||||
# up for the whole gate; restarting it orphans the Agent's namespace (then restart both).
|
||||
mtconnect-gate:
|
||||
image: python:3.13-alpine
|
||||
restart: "no"
|
||||
labels:
|
||||
project: lmxopcua
|
||||
purpose: mtconnect-refused-connection-gate
|
||||
volumes:
|
||||
- ../tests/Drivers/ZB.MOM.WW.OtOpcUa.Driver.MTConnect.IntegrationTests/Docker/adapter.py:/fixtures/adapter.py:ro
|
||||
command: ["python", "-u", "/fixtures/adapter.py", "--host", "0.0.0.0", "--port", "7878"]
|
||||
networks:
|
||||
default:
|
||||
# agent.cfg dials its adapter as `adapter`; inside the shared namespace that resolves here.
|
||||
aliases: [adapter]
|
||||
ports:
|
||||
# Published on the namespace OWNER — the Agent cannot publish ports of its own in this mode.
|
||||
- "5155:5000"
|
||||
|
||||
mtconnect-gate-agent:
|
||||
image: mtconnect/agent:2.7.0.12
|
||||
restart: "no"
|
||||
labels:
|
||||
project: lmxopcua
|
||||
purpose: mtconnect-refused-connection-gate
|
||||
network_mode: "service:mtconnect-gate"
|
||||
depends_on:
|
||||
mtconnect-gate:
|
||||
condition: service_started
|
||||
volumes:
|
||||
- ../tests/Drivers/ZB.MOM.WW.OtOpcUa.Driver.MTConnect.IntegrationTests/Docker/agent.cfg:/mtconnect/config/agent.cfg:ro
|
||||
- ../tests/Drivers/ZB.MOM.WW.OtOpcUa.Driver.MTConnect.IntegrationTests/Docker/Devices.xml:/mtconnect/config/Devices.xml:ro
|
||||
@@ -184,7 +184,7 @@ Read off `MTConnectDriverOptions` / `MTConnectDriverConfigDto`
|
||||
| `SampleIntervalMs` | `1000` | The Agent's `/sample?interval=` query param |
|
||||
| `SampleCount` | `1000` | The Agent's `/sample?count=` query param. **Caps subscription throughput** at `SampleCount / SampleIntervalMs` — see [Subscribe](#subscribe--the-sample-pump) for the adaptive raise |
|
||||
| `HeartbeatMs` | `10000` | The Agent's `/sample?heartbeat=` query param — also the watchdog's liveness window |
|
||||
| `CommsLossConfirmMs` | `2000` | #846: how long the Agent must keep **refusing** connections before every subscribed value goes `BadNoCommunication` (see [Comms loss](#comms-loss-refused-vs-ambiguous-846)). `0` = on the first refusal; negative = fast path off (watchdog only). JSON-only — the AdminUI form preserves it but has no field |
|
||||
| `CommsLossConfirmMs` | `2000` | #846: how long the Agent must keep **refusing** connections before every subscribed value — and, since #852, every native CONDITION (quality only) — goes `BadNoCommunication` (see [Comms loss](#comms-loss-refused-vs-ambiguous-846)). `0` = on the first refusal; negative = fast path off (watchdog only). AdminUI: "Comms-loss confirm (ms)" in the driver modal's Reconnect section (#852; any integer is saved — a negative is never dropped, which would re-enable the 2000 default; ≥ 90000 shows an advisory, as the watchdog always fires first) |
|
||||
| `Probe.Enabled` / `Probe.IntervalMs` / `Probe.TimeoutMs` | `true` / `5000` / `2000` | Background connectivity-probe knobs (mirrors `ModbusProbeOptions`) |
|
||||
| `Reconnect.MinBackoffMs` / `MaxBackoffMs` / `BackoffMultiplier` | `0` / `30000` / `2.0` | Geometric backoff after a failed request or a dropped `/sample` stream |
|
||||
| `Write.Enabled` | `false` | The driver-side half of the write-back opt-in. An absent block means disabled — see [Write-back](#write-back--observation-injection-off-by-default) |
|
||||
@@ -744,16 +744,67 @@ the `/sample` pump now handles it itself:
|
||||
- **Then every subscribed value is published `BadNoCommunication`** (value null) **once per outage**, with one Warning.
|
||||
Health was already Degraded with `LastSuccessfulRead` frozen by the stream failure, so the watchdog still re-initializes
|
||||
at 90 s exactly as before — this only stops stale values reading Good in between. Reads already fall back to `/current`
|
||||
on a degraded stream and fail on their own. Native CONDITIONS are untouched (the host annotates them on the
|
||||
connectivity edge, as before).
|
||||
on a degraded stream and fail on their own.
|
||||
- **Native CONDITIONS too (#852):** at the same moment, every subscribed condition the server holds a statement for gets
|
||||
a quality-only `BadNoCommunication` annotation — exactly what the host's connectivity edge does at the watchdog: Active
|
||||
state, severity, message and acknowledgement untouched, no `/alerts` row, once per outage (a condition already Bad
|
||||
because the Agent reported it UNAVAILABLE is skipped). Recorded in the driver's told record, so recovery lifts it.
|
||||
- **Recovery republishes everything:** the first delivered chunk (a heartbeat counts) or re-baseline after a marked outage
|
||||
republishes every subscribed value from the index — a resumed `/sample` carries only what changed, so an unchanged item
|
||||
would otherwise stay Bad (the (value, status) recovery rule).
|
||||
would otherwise stay Bad (the (value, status) recovery rule). The same delivery re-diffs **every** subscribed condition:
|
||||
an unchanged one gets a quality-only Good (never a re-raise — that would un-acknowledge it); one the Agent still reports
|
||||
UNAVAILABLE stays Bad; one that changed during the outage gets its ordinary Raise/Retrigger/Clear.
|
||||
- Consequence for an Agent **restart** that refuses for longer than the window (the #823 restart refused for ~1 s, inside
|
||||
it): values go Bad for the restart and come back on its first chunk / re-baseline. That is the honest answer — the
|
||||
Agent was down.
|
||||
|
||||
Tests: `MTConnectCommsLossTests` (incl. an ambiguous-failure falsification control and the confirmation window).
|
||||
Tests: `MTConnectCommsLossTests` (incl. an ambiguous-failure falsification control and the confirmation window),
|
||||
`MTConnectConditionCommsLossTests` (#852 — conditions, incl. the UNAVAILABLE and ambiguous-failure controls).
|
||||
|
||||
### Live gate: refused-connection fast path (#846/#852) — recipe
|
||||
|
||||
**Not yet run.** Uses a dedicated, throwaway Agent on the local docker-dev rig — never the shared fixture on
|
||||
`10.100.0.35`. `docker-dev/docker-compose.mtconnect-gate.yml` adds two services to the rig's network: `mtconnect-gate`
|
||||
(the fixture's SHDR adapter, which also owns the network namespace and publishes host `:5155`) and
|
||||
`mtconnect-gate-agent` (`mtconnect/agent:2.7.0.12` with the fixture's `agent.cfg`/`Devices.xml`, running **inside**
|
||||
`mtconnect-gate`'s namespace).
|
||||
|
||||
⚠️ **Why not just `docker stop` an ordinary Agent container:** a stopped compose container also drops out of Docker's
|
||||
embedded DNS, so the driver sees a DNS failure — ambiguous by design, watchdog only — and the gate fails for the wrong
|
||||
reason. With the shared namespace the name `mtconnect-gate` keeps resolving while the Agent is down, and the connect
|
||||
is genuinely refused (smoke-checked: Agent stopped ⇒ `Connection refused` from inside the network; both stopped ⇒
|
||||
`bad address`).
|
||||
|
||||
```bash
|
||||
dcg() { docker compose -f docker-dev/docker-compose.yml -f docker-dev/docker-compose.mtconnect-gate.yml "$@"; } # bash + zsh
|
||||
dcg up -d --no-deps mtconnect-gate mtconnect-gate-agent
|
||||
curl -s http://127.0.0.1:5155/probe | head -3 # Agent answering (Mac side)
|
||||
```
|
||||
|
||||
1. **Driver.** In AdminUI `/raw` (http://localhost:9200, MAIN cluster) add an MTConnect driver, Agent base URI
|
||||
`http://mtconnect-gate:5000`, **Comms-loss confirm (ms) = 2000** (the default — the new field in the Reconnect
|
||||
section), one Device. Browse-commit `fixture_x_pos` (Double) and `fixture_execution` (String), and the CONDITION
|
||||
`fixture_x_travel` with an `alarm` object (optional, for the #852 leg). Optionally reference the two value tags into
|
||||
a UNS equipment. Deploy.
|
||||
2. **Watch.** Client CLI against central-1 (`opc.tcp://localhost:4840`):
|
||||
`dotnet run --project src/Client/ZB.MOM.WW.OtOpcUa.Client.CLI -- subscribe -u opc.tcp://localhost:4840 -n "ns=2;s=<RawPath of fixture_x_pos>" -i 250`
|
||||
(and a second one on the UNS NodeId, `ns=3;s=<Area>/<Line>/<Equipment>/<name>`, if referenced). Values Good and moving.
|
||||
For the condition leg, `alarms -u opc.tcp://localhost:4840 --refresh` in another shell (or read the condition's
|
||||
`Quality` child found with `browse`).
|
||||
3. **Refuse.** Note the wall clock, then `dcg stop mtconnect-gate-agent`.
|
||||
**Expect:** both NodeIds `BadNoCommunication` (value null) **within ~2–3 s** (2000 ms confirmation + the first
|
||||
reconnect attempt), not ~90–118 s; host log `has refused every connection for … ms (definite loss); marking N
|
||||
subscribed value(s) BadNoCommunication`, and with a condition authored `annotated 1 native condition(s)
|
||||
BadNoCommunication`; the condition stays in its Active/acked state. `/hosts` shows the driver Degraded. If you wait
|
||||
past 90 s the #527 watchdog also re-initializes the driver as before — so run step 4 within ~60 s to keep the legs separate.
|
||||
4. **Recover.** `dcg start mtconnect-gate-agent`. **Expect:** values Good again within a few seconds of the
|
||||
Agent answering (first chunk/heartbeat or re-baseline republishes every value, unchanged ones included); the
|
||||
condition's quality Good with no re-raise / no new `/alerts` row. The restarted Agent has a new `instanceId`, so a
|
||||
**RE-BROWSE** chip on `/hosts` is expected and unrelated.
|
||||
5. **Control (optional):** `dcg stop mtconnect-gate-agent mtconnect-gate` — now the name no longer
|
||||
resolves (ambiguous): values must stay Good-but-stale until the ~90 s watchdog. Start `mtconnect-gate` FIRST, then the
|
||||
agent.
|
||||
6. **Clean up:** remove the driver from `/raw` and deploy, then `dcg rm -sf mtconnect-gate-agent mtconnect-gate`.
|
||||
|
||||
### End-to-end gate through a deployed server (#823, 2026-11-11, docker-dev)
|
||||
|
||||
@@ -766,7 +817,7 @@ NodeIds moved together in every leg. Fixture containers were restored healthy af
|
||||
|---|---|
|
||||
| Adapter stop | Both NodeIds `BadNoCommunication` within 1 s of the adapter dying (Agent marks items UNAVAILABLE); driver stayed Healthy. Adapter start ⇒ Good 1–2 s later. |
|
||||
| Agent restart (`docker restart`, ~12 s) | Stream lost and reconnected ~1 s; the stale cursor was refused `OUT_OF_RANGE`, the re-baseline saw the new `instanceId`, **rediscovery raised** — `/hosts` shows the **RE-BROWSE** chip. Values stayed Good (outage shorter than the comms-loss window). |
|
||||
| Agent stopped ~2 min | Values held Good-but-stale; **Bad at 118 s** (comms-loss watchdog, connection refused). Agent start ⇒ re-init on the 10 s retry, Good ~8 s after the Agent was back. **Since #846** a refused connection marks the values `BadNoCommunication` after `CommsLossConfirmMs` (2 s) instead — unit-tested, not yet re-gated live. |
|
||||
| Agent stopped ~2 min | Values held Good-but-stale; **Bad at 118 s** (comms-loss watchdog, connection refused). Agent start ⇒ re-init on the 10 s retry, Good ~8 s after the Agent was back. **Since #846** a refused connection marks the values `BadNoCommunication` after `CommsLossConfirmMs` (2 s) instead — unit-tested, not yet re-gated live (recipe: [Live gate: refused-connection fast path](#live-gate-refused-connection-fast-path-846852--recipe)). |
|
||||
| Agent paused (`docker pause`) | Stream idle watchdog at 24 s (2 × 10000 + 5000); **Bad at 94 s** (watchdog); unpause ⇒ Good within ~3 s. |
|
||||
| Genuine buffer overrun | A stalling TCP proxy froze the driver's transport while 32 000 observations were injected (`AllowPut`) — the Agent's `firstSequence` passed the driver's cursor (75 856 → 77 083). On release the resume was refused `OUT_OF_RANGE`, the driver re-baselined from `/current` within 1 s and resumed; values Good, no restart misdiagnosed. A raised-count stream (count 16, interval 5 s) was **not** lapped by a 266 obs/s burst: chunks are sent back-to-back while data is pending, so `interval` does not cap throughput. |
|
||||
| `SampleCount` vs `bufferSize` (#825) | Test Connect rendered the amber "OK with warning" chip and a deploy logged the Warning (`/hosts` shows `agentBufferSize`). The same gate measured the Agent's real edge — `count = 16384` streams, `16385` is `OUT_OF_RANGE` — so the first build's "at or above" threshold was corrected to "above" (pinned live by `Test_Connect_warns_when_SampleCount_exceeds_the_live_agents_buffer_size`). |
|
||||
|
||||
@@ -92,7 +92,7 @@ internal static class MTConnectConditionAlarmMapper
|
||||
// State untouched; only the quality moves (or is re-asserted on a resync).
|
||||
var next = new MTConnectToldCondition(statement, QualityBad: true);
|
||||
return !bad || resync
|
||||
? (QualityOnly(statement ?? current, current, conditionId, dataItemId, handle, BadNoCommunication), next)
|
||||
? (QualityOnly(statement ?? current, current.TimestampUtc, conditionId, dataItemId, handle, BadNoCommunication), next)
|
||||
: (null, next);
|
||||
}
|
||||
|
||||
@@ -103,7 +103,7 @@ internal static class MTConnectConditionAlarmMapper
|
||||
// Unchanged statement: keep the FIRST text the server was shown. Only a Bad quality is lifted.
|
||||
var kept = told2 with { Statement = statement };
|
||||
return bad
|
||||
? (QualityOnly(statement, current, conditionId, dataItemId, handle, Good), kept)
|
||||
? (QualityOnly(statement, current.TimestampUtc, conditionId, dataItemId, handle, Good), kept)
|
||||
: (null, kept);
|
||||
}
|
||||
|
||||
@@ -123,10 +123,41 @@ internal static class MTConnectConditionAlarmMapper
|
||||
// Normal after nothing / after Normal: nothing to say about the state — the host's connectivity
|
||||
// annotation confirms a materialised condition Good — except lifting a Bad this driver put there.
|
||||
return bad
|
||||
? (QualityOnly(current, current, conditionId, dataItemId, handle, Good), told2)
|
||||
? (QualityOnly(current, current.TimestampUtc, conditionId, dataItemId, handle, Good), told2)
|
||||
: (null, told2);
|
||||
}
|
||||
|
||||
/// <summary>
|
||||
/// lmxopcua#852 — the quality-only <c>BadNoCommunication</c> annotation for a condition whose Agent has been
|
||||
/// confirmed gone (the #846 refused-connection fast path), or <c>null</c> when the server already holds a Bad
|
||||
/// quality for it (or was never told a statement). The told STATEMENT is kept, so the recovery diff that
|
||||
/// follows sees "same statement, quality Bad" and lifts the quality without re-raising — an acked alarm stays
|
||||
/// acked across the outage, exactly as for the host's own connectivity annotation.
|
||||
/// </summary>
|
||||
/// <param name="told">What the server was last told for this condition.</param>
|
||||
/// <param name="conditionId">The condition's ConditionId (RawPath).</param>
|
||||
/// <param name="dataItemId">The Agent DataItem id.</param>
|
||||
/// <param name="handle">The alarm subscription the event belongs to.</param>
|
||||
/// <param name="timestampUtc">When the loss was confirmed.</param>
|
||||
/// <returns>The annotation to raise (or <c>null</c>) and what the server has been told afterwards.</returns>
|
||||
public static (AlarmEventArgs? Event, MTConnectToldCondition Told) CommsLost(
|
||||
MTConnectToldCondition told,
|
||||
string conditionId,
|
||||
string dataItemId,
|
||||
IAlarmSubscriptionHandle handle,
|
||||
DateTime timestampUtc)
|
||||
{
|
||||
ArgumentNullException.ThrowIfNull(told);
|
||||
|
||||
if (told.QualityBad || told.Statement is not { } statement)
|
||||
{
|
||||
return (null, told);
|
||||
}
|
||||
|
||||
return (QualityOnly(statement, timestampUtc, conditionId, dataItemId, handle, BadNoCommunication),
|
||||
told with { QualityBad = true });
|
||||
}
|
||||
|
||||
/// <summary>The Part 9 severity of an active level.</summary>
|
||||
/// <param name="level">An active level.</param>
|
||||
/// <returns>The Fault or Warning severity.</returns>
|
||||
@@ -163,7 +194,7 @@ internal static class MTConnectConditionAlarmMapper
|
||||
/// </summary>
|
||||
private static AlarmEventArgs QualityOnly(
|
||||
MTConnectConditionState shown,
|
||||
MTConnectConditionState current,
|
||||
DateTime sourceTimestampUtc,
|
||||
string conditionId,
|
||||
string dataItemId,
|
||||
IAlarmSubscriptionHandle handle,
|
||||
@@ -178,7 +209,7 @@ internal static class MTConnectConditionAlarmMapper
|
||||
AlarmType: shown.Detail.ConditionType ?? "CONDITION",
|
||||
Message: MessageOf(shown),
|
||||
Severity: severity >= FaultSeverity ? AlarmSeverity.High : AlarmSeverity.Medium,
|
||||
SourceTimestampUtc: current.TimestampUtc,
|
||||
SourceTimestampUtc: sourceTimestampUtc,
|
||||
Kind: AlarmTransitionKind.Unspecified,
|
||||
OpcUaSeverity: severity,
|
||||
QualityStatusCode: quality);
|
||||
|
||||
@@ -36,10 +36,12 @@ namespace ZB.MOM.WW.OtOpcUa.Driver.MTConnect;
|
||||
/// <see cref="_subscriptionLock"/> or the lifecycle semaphore, and never takes either.
|
||||
/// </para>
|
||||
/// <para>
|
||||
/// <b>Comms loss emits nothing here</b>, matching every other native source: a driver that cannot reach
|
||||
/// its Agent has nothing new to say, and the host annotates every condition it owns
|
||||
/// <c>BadNoCommunication</c> on the connectivity edge, keeping each one's Active state. On recovery
|
||||
/// the re-baselined <c>/current</c> is diffed like any other document.
|
||||
/// <b>Comms loss.</b> An AMBIGUOUS loss emits nothing here: the host annotates every condition it owns
|
||||
/// <c>BadNoCommunication</c> on the connectivity edge (the ~90 s watchdog), keeping each one's Active
|
||||
/// state. A CONFIRMED refused-connection loss (#846/#852, <c>MarkConditionsCommsLost</c>) annotates
|
||||
/// every told condition the same way — quality-only, state and ack untouched — at the same moment the data
|
||||
/// values go Bad, and the recovery re-diffs every subscribed condition so the unchanged ones get their
|
||||
/// quality lifted (a resumed <c>/sample</c> carries only what changed).
|
||||
/// </para>
|
||||
/// </remarks>
|
||||
public sealed partial class MTConnectDriver : IAlarmSource
|
||||
@@ -201,20 +203,67 @@ public sealed partial class MTConnectDriver : IAlarmSource
|
||||
|
||||
if (evt is null) continue;
|
||||
|
||||
foreach (var registration in handlers.GetInvocationList())
|
||||
{
|
||||
try
|
||||
{
|
||||
((EventHandler<AlarmEventArgs>)registration).Invoke(this, evt);
|
||||
}
|
||||
catch (Exception ex)
|
||||
{
|
||||
_logger.LogWarning(
|
||||
ex,
|
||||
"MTConnect driver {DriverInstanceId} caught a fault from an alarm subscriber for {ConditionId}; delivery continues.",
|
||||
_driverInstanceId, conditionId);
|
||||
}
|
||||
}
|
||||
RaiseAlarmEvent(handlers, evt, conditionId);
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
/// <summary>
|
||||
/// lmxopcua#852 — the alarm half of the #846 refused-connection fast path: annotates every subscribed
|
||||
/// condition the server holds a Good-quality statement for <c>BadNoCommunication</c> (quality-only — no state,
|
||||
/// severity, message or ack change; no <c>/alerts</c> row) and records that it did, so
|
||||
/// <see cref="EmitConditionTransitions"/> lifts it on recovery. Once per outage per condition: a condition
|
||||
/// already annotated Bad (by this or by an Agent UNAVAILABLE) is skipped. Like every other emit, it records
|
||||
/// nothing when no handler is attached — the host has then detached because it is reconnecting, and its own
|
||||
/// connectivity annotation covers the conditions.
|
||||
/// </summary>
|
||||
/// <returns>How many conditions were annotated.</returns>
|
||||
private int MarkConditionsCommsLost()
|
||||
{
|
||||
var subscription = Volatile.Read(ref _alarmSubscription);
|
||||
if (subscription is not { References.Count: > 0 }) return 0;
|
||||
|
||||
var annotated = 0;
|
||||
lock (_alarmLock)
|
||||
{
|
||||
var handlers = OnAlarmEvent;
|
||||
if (handlers is null) return 0;
|
||||
|
||||
var binding = Volatile.Read(ref _binding);
|
||||
var now = DateTime.UtcNow;
|
||||
|
||||
foreach (var conditionId in subscription.References)
|
||||
{
|
||||
if (!_deliveredConditions.TryGetValue(conditionId, out var told)) continue;
|
||||
|
||||
var (evt, nowTold) = MTConnectConditionAlarmMapper.CommsLost(
|
||||
told, conditionId, binding.ToDataItemId(conditionId), subscription.Handle, now);
|
||||
if (evt is null) continue;
|
||||
|
||||
_deliveredConditions[conditionId] = nowTold;
|
||||
annotated++;
|
||||
RaiseAlarmEvent(handlers, evt, conditionId);
|
||||
}
|
||||
}
|
||||
|
||||
return annotated;
|
||||
}
|
||||
|
||||
/// <summary>Invokes every alarm handler; a faulting one is logged and the next still runs. Caller holds <see cref="_alarmLock"/>.</summary>
|
||||
private void RaiseAlarmEvent(EventHandler<AlarmEventArgs> handlers, AlarmEventArgs evt, string conditionId)
|
||||
{
|
||||
foreach (var registration in handlers.GetInvocationList())
|
||||
{
|
||||
try
|
||||
{
|
||||
((EventHandler<AlarmEventArgs>)registration).Invoke(this, evt);
|
||||
}
|
||||
catch (Exception ex)
|
||||
{
|
||||
_logger.LogWarning(
|
||||
ex,
|
||||
"MTConnect driver {DriverInstanceId} caught a fault from an alarm subscriber for {ConditionId}; delivery continues.",
|
||||
_driverInstanceId, conditionId);
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
@@ -30,12 +30,14 @@ namespace ZB.MOM.WW.OtOpcUa.Driver.MTConnect;
|
||||
/// is already Degraded with <c>LastSuccessfulRead</c> frozen (the stream failure degraded it), so the watchdog
|
||||
/// still re-initializes at 90 s exactly as before — this path only stops serving stale values as Good in the
|
||||
/// meantime. Reads already fall back to <c>/current</c> on a degraded stream and fail Bad on their own.
|
||||
/// Native CONDITIONS are not touched: the host annotates them on the connectivity edge, as before.
|
||||
/// Native CONDITIONS (#852) get what the host's connectivity edge gives them: a quality-only
|
||||
/// <c>BadNoCommunication</c> annotation, Active state and ack untouched (<c>MarkConditionsCommsLost</c>).
|
||||
/// </para>
|
||||
/// <para>
|
||||
/// <b>Recovery republishes everything.</b> The first delivered chunk or re-baseline after a marked outage
|
||||
/// republishes every subscribed value from the index — a resumed <c>/sample</c> only carries what CHANGED, so
|
||||
/// an unchanged data item would otherwise stay Bad until it next moved (the (value, status) recovery rule).
|
||||
/// Conditions are re-diffed in full for the same reason, which lifts each annotated one's quality.
|
||||
/// </para>
|
||||
/// </remarks>
|
||||
public sealed partial class MTConnectDriver
|
||||
@@ -147,6 +149,16 @@ public sealed partial class MTConnectDriver
|
||||
"MTConnect driver {DriverInstanceId}: {AgentUri} has refused every connection for {RefusedMs} ms (definite loss); marking {Count} subscribed value(s) BadNoCommunication until it answers again.",
|
||||
_driverInstanceId, AgentDisplay(options), (long)refusedFor.TotalMilliseconds, references.Count);
|
||||
FanOutStatus(references, StatusBadNoCommunication);
|
||||
|
||||
// #852 — and the native conditions, quality-only (state + ack kept), as the host's connectivity edge does.
|
||||
var conditions = MarkConditionsCommsLost();
|
||||
if (conditions > 0)
|
||||
{
|
||||
_logger.LogWarning(
|
||||
"MTConnect driver {DriverInstanceId}: annotated {Count} native condition(s) BadNoCommunication (quality only; state and acknowledgement kept).",
|
||||
_driverInstanceId, conditions);
|
||||
}
|
||||
|
||||
return true;
|
||||
}
|
||||
|
||||
@@ -170,6 +182,10 @@ public sealed partial class MTConnectDriver
|
||||
"MTConnect driver {DriverInstanceId}: the Agent answered again; republishing every subscribed value.",
|
||||
_driverInstanceId);
|
||||
FanOut(SubscribedReferences(), ObservationIndex);
|
||||
|
||||
// #852 — every subscribed condition, not just the ones this delivery touched: an unchanged condition the
|
||||
// outage annotated Bad gets a quality-only Good; one still UNAVAILABLE at the Agent stays Bad.
|
||||
EmitConditionTransitions(touchedDataItemIds: null, resync: false);
|
||||
}
|
||||
|
||||
/// <summary>Publishes a value-less <paramref name="status"/> to every handle subscribing each reference.</summary>
|
||||
|
||||
+38
-3
@@ -107,6 +107,16 @@
|
||||
<InputNumber @bind-Value="_form.ReconnectBackoffMultiplier" @bind-Value:after="EmitAsync" class="form-control form-control-sm" />
|
||||
<div class="form-text">Default 2.0 (doubles each retry).</div>
|
||||
</div>
|
||||
<div class="col-md-3">
|
||||
<label class="form-label">Comms-loss confirm (ms)</label>
|
||||
<InputNumber @bind-Value="_form.CommsLossConfirmMs" @bind-Value:after="EmitAsync" class="form-control form-control-sm" />
|
||||
<div class="form-text">
|
||||
Default 2000. How long the Agent must keep <strong>refusing</strong> connections before every
|
||||
subscribed value and native condition goes <span class="mono">BadNoCommunication</span>.
|
||||
<span class="mono">0</span> = on the first refusal; negative = fast path off (only the ~90 s
|
||||
comms-loss watchdog). Timeouts and a paused Agent always wait for the watchdog.
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
</section>
|
||||
@@ -299,6 +309,12 @@
|
||||
/// <summary>Multiplier applied to the reconnect backoff each retry.</summary>
|
||||
public double ReconnectBackoffMultiplier { get; set; } = 2.0;
|
||||
/// <summary>
|
||||
/// #846/#852 — how long refused connections must persist before values and native conditions go
|
||||
/// <c>BadNoCommunication</c>. Every integer is meaningful to the driver: <c>0</c> = the first refusal,
|
||||
/// negative = the fast path is off (watchdog only). So, unlike the timing knobs, it is always written.
|
||||
/// </summary>
|
||||
public int CommsLossConfirmMs { get; set; } = DefaultCommsLossConfirmMs;
|
||||
/// <summary>
|
||||
/// Whether the driver's half of the write-back gate is armed. Default <c>false</c> — the
|
||||
/// driver shipped read-only, and the Agent's own <c>AllowPut</c> is an independent second
|
||||
/// gate that this cannot arm.
|
||||
@@ -329,6 +345,7 @@
|
||||
ReconnectMinBackoffMs = dto.Reconnect?.MinBackoffMs ?? defaults.ReconnectMinBackoffMs,
|
||||
ReconnectMaxBackoffMs = dto.Reconnect?.MaxBackoffMs ?? defaults.ReconnectMaxBackoffMs,
|
||||
ReconnectBackoffMultiplier = dto.Reconnect?.BackoffMultiplier ?? defaults.ReconnectBackoffMultiplier,
|
||||
CommsLossConfirmMs = dto.CommsLossConfirmMs ?? defaults.CommsLossConfirmMs,
|
||||
WriteEnabled = dto.Write?.Enabled ?? defaults.WriteEnabled,
|
||||
};
|
||||
}
|
||||
@@ -344,6 +361,9 @@
|
||||
SampleIntervalMs = Positive(SampleIntervalMs),
|
||||
SampleCount = Positive(SampleCount),
|
||||
HeartbeatMs = Positive(HeartbeatMs),
|
||||
// Written as-is, negative included: a negative is "off", not an unusable value to drop — dropping it
|
||||
// would silently re-enable the fast path at the driver's 2000 ms default.
|
||||
CommsLossConfirmMs = CommsLossConfirmMs,
|
||||
Probe = new MTConnectProbeConfigDto
|
||||
{
|
||||
Enabled = ProbeEnabled,
|
||||
@@ -383,11 +403,24 @@
|
||||
if (ProbeEnabled && ProbeTimeoutMs <= 0) { bad.Add("Probe timeout"); }
|
||||
if (ReconnectMaxBackoffMs <= 0) { bad.Add("Max backoff"); }
|
||||
|
||||
return bad.Count == 0
|
||||
? null
|
||||
: $"Must be greater than zero: {string.Join(", ", bad)}. A non-positive value is dropped on save and the driver's default applies.";
|
||||
if (bad.Count > 0)
|
||||
{
|
||||
return $"Must be greater than zero: {string.Join(", ", bad)}. A non-positive value is dropped on save and the driver's default applies.";
|
||||
}
|
||||
|
||||
// Advisory only (the value is still saved): the runtime's comms-loss watchdog re-initializes a driver
|
||||
// whose reads have been frozen for 90 s, so a window that long can never fire first.
|
||||
return CommsLossConfirmMs >= CommsLossWatchdogMs
|
||||
? $"Comms-loss confirm of {CommsLossConfirmMs} ms is at or beyond the ~{CommsLossWatchdogMs / 1000} s comms-loss watchdog, so the fast path never fires before it. Use a negative value to turn the fast path off explicitly."
|
||||
: null;
|
||||
}
|
||||
|
||||
/// <summary>The driver's own default for <see cref="CommsLossConfirmMs"/> (<c>MTConnectDriverOptions</c>).</summary>
|
||||
public const int DefaultCommsLossConfirmMs = 2_000;
|
||||
|
||||
/// <summary>The runtime's hardcoded comms-loss watchdog threshold (<c>DefaultCommsLossTimeout</c>, 90 s).</summary>
|
||||
public const int CommsLossWatchdogMs = 90_000;
|
||||
|
||||
private static int? Positive(int value) => value > 0 ? value : null;
|
||||
|
||||
private static string? Trimmed(string? value)
|
||||
@@ -418,6 +451,8 @@
|
||||
public int? SampleCount { get; init; }
|
||||
/// <summary>The <c>/sample</c> <c>heartbeat</c> query parameter in ms.</summary>
|
||||
public int? HeartbeatMs { get; init; }
|
||||
/// <summary>Refused-connection confirmation window in ms (#846; 0 = first refusal, negative = off).</summary>
|
||||
public int? CommsLossConfirmMs { get; init; }
|
||||
/// <summary>Background connectivity-probe knobs.</summary>
|
||||
public MTConnectProbeConfigDto? Probe { get; init; }
|
||||
/// <summary>Reconnect backoff knobs.</summary>
|
||||
|
||||
+233
@@ -0,0 +1,233 @@
|
||||
using System.Collections.Concurrent;
|
||||
using System.Net.Http;
|
||||
using System.Net.Sockets;
|
||||
using Shouldly;
|
||||
using Xunit;
|
||||
using ZB.MOM.WW.OtOpcUa.Core.Abstractions;
|
||||
|
||||
namespace ZB.MOM.WW.OtOpcUa.Driver.MTConnect.Tests;
|
||||
|
||||
/// <summary>
|
||||
/// lmxopcua#852 — the #846 refused-connection fast path reaches native CONDITIONS too: once the refusals are
|
||||
/// confirmed every told condition gets a quality-only <c>BadNoCommunication</c> (state and ack untouched, no
|
||||
/// Raise/Clear), once per outage; the first delivery afterwards lifts it with a quality-only Good — even when that
|
||||
/// delivery does not mention the condition. Ambiguous failures stay with the watchdog.
|
||||
/// </summary>
|
||||
/// <remarks>Every absence is followed by an event that MUST arrive, asserted to be the next one.</remarks>
|
||||
public sealed class MTConnectConditionCommsLossTests
|
||||
{
|
||||
private const string ConditionRawPath = "Plant/MT/dev1/SystemCond";
|
||||
private const string ConditionId = "dev1_system_cond";
|
||||
private const long Instance = 1655000000L;
|
||||
private const uint BadNoCommunication = 0x80310000u;
|
||||
|
||||
private static readonly DateTime ObservedAt = new(2026, 11, 11, 12, 0, 0, DateTimeKind.Utc);
|
||||
private static readonly TimeSpan Watchdog = TimeSpan.FromSeconds(30);
|
||||
|
||||
private static CancellationToken Ct => TestContext.Current.CancellationToken;
|
||||
|
||||
private static Exception Refused() =>
|
||||
new HttpRequestException("Connection refused (fixture-agent:5000)", new SocketException((int)SocketError.ConnectionRefused));
|
||||
|
||||
private static MTConnectStreamsResult Chunk(long first, params MTConnectObservation[] observations) =>
|
||||
new(Instance, first + 1, first, observations);
|
||||
|
||||
private static MTConnectObservation Cond(string state, string? nativeCode = null, string? text = null) =>
|
||||
new(ConditionId, state, ObservedAt, Condition: new MTConnectConditionDetail(text, nativeCode, null, null, "SYSTEM"));
|
||||
|
||||
private static async Task<(MTConnectDriver Driver, CannedAgentClient Agent, ConcurrentQueue<AlarmEventArgs> Events)>
|
||||
SubscribedAsync()
|
||||
{
|
||||
var agent = CannedAgentClient.FromFixtures();
|
||||
var driver = new MTConnectDriver(
|
||||
new MTConnectDriverOptions
|
||||
{
|
||||
AgentUri = "http://fixture-agent:5000",
|
||||
RequestTimeoutMs = 5000,
|
||||
CommsLossConfirmMs = 0,
|
||||
Reconnect = new MTConnectReconnectOptions { MinBackoffMs = 0, MaxBackoffMs = 200 },
|
||||
RawTags = [new RawTagEntry(ConditionRawPath, $$"""{"address":"{{ConditionId}}"}""", false, "dev1")],
|
||||
},
|
||||
"mt-852",
|
||||
_ => agent);
|
||||
await driver.InitializeAsync("{}", Ct);
|
||||
|
||||
var events = new ConcurrentQueue<AlarmEventArgs>();
|
||||
driver.OnAlarmEvent += (_, e) => events.Enqueue(e);
|
||||
await driver.SubscribeAlarmsAsync([ConditionRawPath], Ct);
|
||||
await agent.WaitForSampleCallsAsync(1).WaitAsync(Watchdog, Ct);
|
||||
return (driver, agent, events);
|
||||
}
|
||||
|
||||
private static async Task WaitUntil(Func<bool> condition, int timeoutMs = 10_000)
|
||||
{
|
||||
var deadline = DateTime.UtcNow.AddMilliseconds(timeoutMs);
|
||||
while (!condition())
|
||||
{
|
||||
if (DateTime.UtcNow > deadline) throw new TimeoutException("condition not reached");
|
||||
await Task.Delay(20, Ct);
|
||||
}
|
||||
}
|
||||
|
||||
private static async Task RecoverAsync(CannedAgentClient agent)
|
||||
{
|
||||
agent.SampleFailure = null;
|
||||
var calls = agent.SampleCallCount;
|
||||
await agent.WaitForSampleCallsAsync(calls + 1).WaitAsync(Watchdog, Ct);
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public async Task A_refused_agent_annotates_an_active_condition_bad_once_and_the_first_delivery_lifts_it()
|
||||
{
|
||||
var (driver, agent, events) = await SubscribedAsync();
|
||||
try
|
||||
{
|
||||
await agent.PumpAsync(Chunk(108, Cond("Fault", "E17", "overtemp"))).WaitAsync(Watchdog, Ct);
|
||||
events.ShouldHaveSingleItem().Kind.ShouldBe(AlarmTransitionKind.Raise);
|
||||
|
||||
agent.SampleFailure = Refused();
|
||||
agent.EndStream();
|
||||
await WaitUntil(() => events.Count == 2);
|
||||
var bad = events.Last();
|
||||
bad.IsQualityOnly.ShouldBeTrue("the Active state and the ack must not move — quality only");
|
||||
bad.QualityStatusCode.ShouldBe(BadNoCommunication);
|
||||
bad.ConditionId.ShouldBe(ConditionRawPath);
|
||||
bad.OpcUaSeverity.ShouldBe(800);
|
||||
driver.CommsLossMarked.ShouldBeTrue();
|
||||
|
||||
await Task.Delay(400, Ct); // more refusals: annotated ONCE per outage
|
||||
events.Count.ShouldBe(2);
|
||||
|
||||
// The Agent is back; an observation-free heartbeat does not mention the condition, and must still lift it.
|
||||
await RecoverAsync(agent);
|
||||
await agent.PumpAsync(Chunk(109)).WaitAsync(Watchdog, Ct);
|
||||
await WaitUntil(() => events.Count == 3);
|
||||
var good = events.Last();
|
||||
good.IsQualityOnly.ShouldBeTrue("an unchanged Fault is not re-raised (that would un-acknowledge it)");
|
||||
good.QualityStatusCode.ShouldBe(0u);
|
||||
|
||||
// Positive control for "nothing else was emitted": the next real transition is the next event.
|
||||
await agent.PumpAsync(Chunk(110, Cond("Normal"))).WaitAsync(Watchdog, Ct);
|
||||
await WaitUntil(() => events.Count == 4);
|
||||
events.Last().Kind.ShouldBe(AlarmTransitionKind.Clear);
|
||||
}
|
||||
finally
|
||||
{
|
||||
await driver.ShutdownAsync(CancellationToken.None);
|
||||
}
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public async Task An_inactive_told_condition_is_annotated_too()
|
||||
{
|
||||
var (driver, agent, events) = await SubscribedAsync();
|
||||
try
|
||||
{
|
||||
// Raise + Clear: the server holds a told Normal statement for the condition.
|
||||
await agent.PumpAsync(Chunk(108, Cond("Fault", "E17"))).WaitAsync(Watchdog, Ct);
|
||||
await agent.PumpAsync(Chunk(109, Cond("Normal"))).WaitAsync(Watchdog, Ct);
|
||||
events.Count.ShouldBe(2);
|
||||
|
||||
agent.SampleFailure = Refused();
|
||||
agent.EndStream();
|
||||
await WaitUntil(() => events.Count == 3);
|
||||
events.Last().IsQualityOnly.ShouldBeTrue();
|
||||
events.Last().QualityStatusCode.ShouldBe(BadNoCommunication);
|
||||
|
||||
await RecoverAsync(agent);
|
||||
await agent.PumpAsync(Chunk(110)).WaitAsync(Watchdog, Ct);
|
||||
await WaitUntil(() => events.Count == 4);
|
||||
events.Last().IsQualityOnly.ShouldBeTrue();
|
||||
events.Last().QualityStatusCode.ShouldBe(0u);
|
||||
}
|
||||
finally
|
||||
{
|
||||
await driver.ShutdownAsync(CancellationToken.None);
|
||||
}
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public async Task A_condition_already_unavailable_is_not_re_annotated_and_stays_bad_on_recovery()
|
||||
{
|
||||
var (driver, agent, events) = await SubscribedAsync();
|
||||
try
|
||||
{
|
||||
await agent.PumpAsync(Chunk(108, Cond("Fault", "E17"))).WaitAsync(Watchdog, Ct);
|
||||
await agent.PumpAsync(Chunk(109, Cond("UNAVAILABLE"))).WaitAsync(Watchdog, Ct);
|
||||
events.Count.ShouldBe(2);
|
||||
events.Last().QualityStatusCode.ShouldBe(BadNoCommunication);
|
||||
|
||||
agent.SampleFailure = Refused();
|
||||
agent.EndStream();
|
||||
await WaitUntil(() => driver.CommsLossMarked);
|
||||
|
||||
// The Agent answers, but the adapter is still UNAVAILABLE: no Good may be told.
|
||||
await RecoverAsync(agent);
|
||||
await agent.PumpAsync(Chunk(110)).WaitAsync(Watchdog, Ct);
|
||||
await WaitUntil(() => !driver.CommsLossMarked);
|
||||
|
||||
// Positive control: the adapter returns with the same Fault — quality lifted, NOT re-raised — and that
|
||||
// is the very next event after the UNAVAILABLE annotation.
|
||||
await agent.PumpAsync(Chunk(111, Cond("Fault", "E17"))).WaitAsync(Watchdog, Ct);
|
||||
await WaitUntil(() => events.Count >= 3);
|
||||
events.Count.ShouldBe(3);
|
||||
events.Last().IsQualityOnly.ShouldBeTrue();
|
||||
events.Last().QualityStatusCode.ShouldBe(0u);
|
||||
}
|
||||
finally
|
||||
{
|
||||
await driver.ShutdownAsync(CancellationToken.None);
|
||||
}
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public async Task An_ambiguous_failure_annotates_no_condition()
|
||||
{
|
||||
var (driver, agent, events) = await SubscribedAsync();
|
||||
try
|
||||
{
|
||||
await agent.PumpAsync(Chunk(108, Cond("Fault", "E17"))).WaitAsync(Watchdog, Ct);
|
||||
events.Count.ShouldBe(1);
|
||||
|
||||
agent.SampleFailure = new TimeoutException("the Agent did not answer /sample in time");
|
||||
var calls = agent.SampleCallCount;
|
||||
agent.EndStream();
|
||||
await agent.WaitForSampleCallsAsync(calls + 3).WaitAsync(Watchdog, Ct);
|
||||
driver.CommsLossMarked.ShouldBeFalse();
|
||||
|
||||
// Positive control: the next event is the real Clear, not a quality annotation.
|
||||
await RecoverAsync(agent);
|
||||
await agent.PumpAsync(Chunk(109, Cond("Normal"))).WaitAsync(Watchdog, Ct);
|
||||
await WaitUntil(() => events.Count >= 2);
|
||||
events.Count.ShouldBe(2);
|
||||
events.Last().Kind.ShouldBe(AlarmTransitionKind.Clear);
|
||||
}
|
||||
finally
|
||||
{
|
||||
await driver.ShutdownAsync(CancellationToken.None);
|
||||
}
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public void CommsLost_keeps_the_told_statement_and_says_nothing_twice()
|
||||
{
|
||||
var handle = new MTConnectAlarmSubscriptionHandle(1);
|
||||
var warning = new MTConnectConditionState(
|
||||
MTConnectConditionLevel.Warning, new MTConnectConditionDetail("coolant", "W1", null, null, "SYSTEM"), ObservedAt, 1);
|
||||
var told = new MTConnectToldCondition(warning, QualityBad: false);
|
||||
var at = ObservedAt.AddMinutes(5);
|
||||
|
||||
var (evt, after) = MTConnectConditionAlarmMapper.CommsLost(told, ConditionRawPath, ConditionId, handle, at);
|
||||
|
||||
evt.ShouldNotBeNull();
|
||||
evt!.IsQualityOnly.ShouldBeTrue();
|
||||
evt.QualityStatusCode.ShouldBe(BadNoCommunication);
|
||||
evt.SourceTimestampUtc.ShouldBe(at);
|
||||
evt.OpcUaSeverity.ShouldBe(500);
|
||||
after.ShouldBe(told with { QualityBad = true });
|
||||
|
||||
MTConnectConditionAlarmMapper.CommsLost(after, ConditionRawPath, ConditionId, handle, at).Event.ShouldBeNull();
|
||||
MTConnectConditionAlarmMapper.CommsLost(
|
||||
new MTConnectToldCondition(null, QualityBad: true), ConditionRawPath, ConditionId, handle, at).Event.ShouldBeNull();
|
||||
}
|
||||
}
|
||||
@@ -394,7 +394,7 @@ public sealed class MTConnectFactoryTests
|
||||
public void The_document_the_AdminUI_form_emits_arms_write_back()
|
||||
{
|
||||
const string formEmitted =
|
||||
"""{"agentUri":"http://agent:5000","requestTimeoutMs":5000,"sampleIntervalMs":1000,"sampleCount":1000,"heartbeatMs":10000,"probe":{"enabled":true,"intervalMs":5000,"timeoutMs":2000},"reconnect":{"minBackoffMs":0,"maxBackoffMs":30000,"backoffMultiplier":2},"write":{"enabled":true}}""";
|
||||
"""{"agentUri":"http://agent:5000","requestTimeoutMs":5000,"sampleIntervalMs":1000,"sampleCount":1000,"heartbeatMs":10000,"commsLossConfirmMs":2000,"probe":{"enabled":true,"intervalMs":5000,"timeoutMs":2000},"reconnect":{"minBackoffMs":0,"maxBackoffMs":30000,"backoffMultiplier":2},"write":{"enabled":true}}""";
|
||||
|
||||
var options = MTConnectDriver.ParseOptions("mt1", formEmitted);
|
||||
|
||||
@@ -407,8 +407,29 @@ public sealed class MTConnectFactoryTests
|
||||
"mt1",
|
||||
formEmitted.Replace("""enabled":true}""", """enabled":false}""", StringComparison.Ordinal))
|
||||
.Write.Enabled.ShouldBeFalse();
|
||||
options.CommsLossConfirmMs.ShouldBe(2000);
|
||||
}
|
||||
|
||||
/// <summary>
|
||||
/// lmxopcua#852: the AdminUI form's <c>commsLossConfirmMs</c> field binds through this parse as a plain
|
||||
/// integer in every meaningful state — default, 0 (first refusal) and negative (fast path off). The literals are
|
||||
/// the form's own output (byte-pinned on the form side by
|
||||
/// <c>MTConnectDriverFormModelTests.The_comms_loss_documents_are_the_literals_the_driver_test_parses</c>); an
|
||||
/// absent key keeps the driver default.
|
||||
/// </summary>
|
||||
[Theory]
|
||||
[InlineData(CommsLossOffDocument, -1)]
|
||||
[InlineData(CommsLossImmediateDocument, 0)]
|
||||
[InlineData("""{"agentUri":"http://agent:5000"}""", 2000)]
|
||||
public void The_comms_loss_window_the_AdminUI_form_emits_binds(string formEmitted, int expected) =>
|
||||
MTConnectDriver.ParseOptions("mt1", formEmitted).CommsLossConfirmMs.ShouldBe(expected);
|
||||
|
||||
private const string CommsLossOffDocument =
|
||||
"""{"agentUri":"http://agent:5000","requestTimeoutMs":5000,"sampleIntervalMs":1000,"sampleCount":1000,"heartbeatMs":10000,"commsLossConfirmMs":-1,"probe":{"enabled":true,"intervalMs":5000,"timeoutMs":2000},"reconnect":{"minBackoffMs":0,"maxBackoffMs":30000,"backoffMultiplier":2},"write":{"enabled":false}}""";
|
||||
|
||||
private const string CommsLossImmediateDocument =
|
||||
"""{"agentUri":"http://agent:5000","requestTimeoutMs":5000,"sampleIntervalMs":1000,"sampleCount":1000,"heartbeatMs":10000,"commsLossConfirmMs":0,"probe":{"enabled":true,"intervalMs":5000,"timeoutMs":2000},"reconnect":{"minBackoffMs":0,"maxBackoffMs":30000,"backoffMultiplier":2},"write":{"enabled":false}}""";
|
||||
|
||||
/// <summary>
|
||||
/// A malformed <c>write</c> block fails <b>exactly as its siblings do</b> — the whole
|
||||
/// document is one <c>JsonSerializer.Deserialize</c>, so a non-boolean <c>enabled</c> faults
|
||||
|
||||
@@ -200,6 +200,75 @@ public sealed class MTConnectDriverFormModelTests
|
||||
form.WriteEnabled.ShouldBeTrue();
|
||||
}
|
||||
|
||||
// ---- comms-loss confirmation window (#846/#852) --------------------------------------------
|
||||
|
||||
[Fact]
|
||||
public void Comms_loss_window_defaults_to_the_drivers_2000_and_an_absent_key_loads_as_it()
|
||||
{
|
||||
new MTConnectDriverForm.MTConnectFormModel().CommsLossConfirmMs.ShouldBe(2000);
|
||||
MTConnectDriverForm.MTConnectFormModel.FromJson("""{"agentUri":"http://a:5000"}""")
|
||||
.CommsLossConfirmMs.ShouldBe(2000);
|
||||
}
|
||||
|
||||
[Theory]
|
||||
[InlineData(-1)] // fast path off — must NOT be dropped like a non-positive timing knob, or it re-enables at 2000
|
||||
[InlineData(0)] // mark on the first refusal
|
||||
[InlineData(5000)]
|
||||
public void Every_comms_loss_window_is_written_as_a_number_and_reloads(int value)
|
||||
{
|
||||
var o = Emit(new MTConnectDriverForm.MTConnectFormModel { AgentUri = "http://agent:5000", CommsLossConfirmMs = value });
|
||||
|
||||
// A JSON NUMBER (the driver's DTO is int?) — the class of bug where a form emitted a shape the factory DTO
|
||||
// could not bind.
|
||||
o["commsLossConfirmMs"]!.GetValueKind().ShouldBe(JsonValueKind.Number);
|
||||
o["commsLossConfirmMs"]!.GetValue<int>().ShouldBe(value);
|
||||
MTConnectDriverForm.MTConnectFormModel.FromJson(o.ToJsonString()).CommsLossConfirmMs.ShouldBe(value);
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public void A_hand_authored_comms_loss_window_loads_from_pascal_case_and_survives_an_unrelated_edit()
|
||||
{
|
||||
// The pre-#852 JSON-only path (docs/drivers/MTConnect.md shows the PascalCase spelling).
|
||||
const string existing = """{"AgentUri":"http://a:5000","CommsLossConfirmMs":-5,"tags":[{"fullName":"x1"}]}""";
|
||||
|
||||
var form = MTConnectDriverForm.MTConnectFormModel.FromJson(existing);
|
||||
form.CommsLossConfirmMs.ShouldBe(-5);
|
||||
form.SampleCount = 250;
|
||||
|
||||
var o = Emit(form, existing);
|
||||
o["commsLossConfirmMs"]!.GetValue<int>().ShouldBe(-5);
|
||||
o["tags"]!.AsArray().Count.ShouldBe(1);
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public void Validate_accepts_negative_and_zero_windows_and_warns_at_the_watchdog()
|
||||
{
|
||||
new MTConnectDriverForm.MTConnectFormModel { AgentUri = "http://a:5000", CommsLossConfirmMs = -1 }.Validate().ShouldBeNull();
|
||||
new MTConnectDriverForm.MTConnectFormModel { AgentUri = "http://a:5000", CommsLossConfirmMs = 0 }.Validate().ShouldBeNull();
|
||||
new MTConnectDriverForm.MTConnectFormModel { AgentUri = "http://a:5000", CommsLossConfirmMs = 89_999 }.Validate().ShouldBeNull();
|
||||
new MTConnectDriverForm.MTConnectFormModel { AgentUri = "http://a:5000", CommsLossConfirmMs = 90_000 }
|
||||
.Validate().ShouldNotBeNull().ShouldContain("watchdog");
|
||||
}
|
||||
|
||||
[Theory]
|
||||
[InlineData(-1, CommsLossOffDocument)]
|
||||
[InlineData(0, CommsLossImmediateDocument)]
|
||||
public void The_comms_loss_documents_are_the_literals_the_driver_test_parses(int value, string expected)
|
||||
{
|
||||
// Mirrored in MTConnectFactoryTests.The_comms_loss_window_the_AdminUI_form_emits_binds, which runs these exact
|
||||
// bytes through MTConnectDriver.ParseOptions — the driver's real options DTO.
|
||||
MTConnectDriverForm.BuildConfigJson(
|
||||
new MTConnectDriverForm.MTConnectFormModel { AgentUri = "http://agent:5000", CommsLossConfirmMs = value },
|
||||
null)
|
||||
.ShouldBe(expected);
|
||||
}
|
||||
|
||||
private const string CommsLossOffDocument =
|
||||
"""{"agentUri":"http://agent:5000","requestTimeoutMs":5000,"sampleIntervalMs":1000,"sampleCount":1000,"heartbeatMs":10000,"commsLossConfirmMs":-1,"probe":{"enabled":true,"intervalMs":5000,"timeoutMs":2000},"reconnect":{"minBackoffMs":0,"maxBackoffMs":30000,"backoffMultiplier":2},"write":{"enabled":false}}""";
|
||||
|
||||
private const string CommsLossImmediateDocument =
|
||||
"""{"agentUri":"http://agent:5000","requestTimeoutMs":5000,"sampleIntervalMs":1000,"sampleCount":1000,"heartbeatMs":10000,"commsLossConfirmMs":0,"probe":{"enabled":true,"intervalMs":5000,"timeoutMs":2000},"reconnect":{"minBackoffMs":0,"maxBackoffMs":30000,"backoffMultiplier":2},"write":{"enabled":false}}""";
|
||||
|
||||
[Fact]
|
||||
public void The_write_enabled_document_is_the_literal_the_driver_test_parses()
|
||||
{
|
||||
@@ -215,7 +284,7 @@ public sealed class MTConnectDriverFormModelTests
|
||||
|
||||
/// <summary>The form's output for a write-enabled instance, mirrored in the driver test suite.</summary>
|
||||
private const string WriteEnabledDocument =
|
||||
"""{"agentUri":"http://agent:5000","requestTimeoutMs":5000,"sampleIntervalMs":1000,"sampleCount":1000,"heartbeatMs":10000,"probe":{"enabled":true,"intervalMs":5000,"timeoutMs":2000},"reconnect":{"minBackoffMs":0,"maxBackoffMs":30000,"backoffMultiplier":2},"write":{"enabled":true}}""";
|
||||
"""{"agentUri":"http://agent:5000","requestTimeoutMs":5000,"sampleIntervalMs":1000,"sampleCount":1000,"heartbeatMs":10000,"commsLossConfirmMs":2000,"probe":{"enabled":true,"intervalMs":5000,"timeoutMs":2000},"reconnect":{"minBackoffMs":0,"maxBackoffMs":30000,"backoffMultiplier":2},"write":{"enabled":true}}""";
|
||||
|
||||
// ---- Validate ------------------------------------------------------------------------------
|
||||
|
||||
@@ -263,6 +332,7 @@ public sealed class MTConnectDriverFormModelTests
|
||||
ReconnectMinBackoffMs = 100,
|
||||
ReconnectMaxBackoffMs = 60000,
|
||||
ReconnectBackoffMultiplier = 1.5,
|
||||
CommsLossConfirmMs = -1,
|
||||
};
|
||||
|
||||
var json = MTConnectDriverForm.BuildConfigJson(original, null);
|
||||
@@ -280,6 +350,7 @@ public sealed class MTConnectDriverFormModelTests
|
||||
back.ReconnectMinBackoffMs.ShouldBe(100);
|
||||
back.ReconnectMaxBackoffMs.ShouldBe(60000);
|
||||
back.ReconnectBackoffMultiplier.ShouldBe(1.5);
|
||||
back.CommsLossConfirmMs.ShouldBe(-1);
|
||||
}
|
||||
|
||||
[Fact]
|
||||
@@ -373,6 +444,7 @@ public sealed class MTConnectDriverFormModelTests
|
||||
dto.SampleIntervalMs.ShouldBe(1000);
|
||||
dto.SampleCount.ShouldBe(1000);
|
||||
dto.HeartbeatMs.ShouldBe(10000);
|
||||
dto.CommsLossConfirmMs.ShouldBe(2000);
|
||||
dto.Probe.ShouldNotBeNull();
|
||||
dto.Probe!.Enabled.ShouldBe(true);
|
||||
dto.Probe.IntervalMs.ShouldBe(5000);
|
||||
@@ -393,6 +465,7 @@ public sealed class MTConnectDriverFormModelTests
|
||||
public int? SampleIntervalMs { get; init; }
|
||||
public int? SampleCount { get; init; }
|
||||
public int? HeartbeatMs { get; init; }
|
||||
public int? CommsLossConfirmMs { get; init; }
|
||||
public WireProbe? Probe { get; init; }
|
||||
public WireReconnect? Reconnect { get; init; }
|
||||
public WireWrite? Write { get; init; }
|
||||
|
||||
Reference in New Issue
Block a user