90bdaa4437b72f26f61f9377f49c9bc8c19ffb44
2895 Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 Joseph Doherty
|
549c656489 |
fix(alarms): re-assert non-normal scripted-alarm conditions after a (re)load (#487)
A deploy that triggers a FULL address-space rebuild clears `_alarmConditions` and re-materialises every condition node fresh — inactive, acked, confirmed, unshelved. The engine reloads its persisted state and re-derives Active from the predicate, so it ends up correct; but there is no transition to report, so `LoadAsync` yields `EmissionKind.None` and `OnEngineEmission` drops it. Nothing writes the node. Live-reproduced on the docker-dev rig against the pre-fix image: an alarm that had fired and cleared but was still UNACKNOWLEDGED came back from a rebuild reporting Acknowledged with Retain=false — so it disappeared from ConditionRefresh entirely, while the engine still held Unacknowledged. An operator's outstanding alarm silently vanishes from the alarm list on deploy. `ScriptedAlarmHostActor.ReassertConditionNodes` closes it. After each load it reads the new `ScriptedAlarmEngine.GetProjections()` and sends one `AlarmStateUpdate` per alarm NOT in the Part 9 no-event position. Load-bearing properties: - Node-only. It sends `AlarmStateUpdate` and nothing else — never the `alerts` topic, never the telemetry hub. `alerts` is the historization path, so a row per alarm per deploy would append a duplicate historian/AVEVA record every time anyone deploys. This is why it reads a projection rather than re-emitting: `ScriptedAlarmProjection` carries no `EmissionKind`, so there is nothing for the emission machinery — or a future refactor — to mistake for a transition. The issue's sketch said `GetState(alarmId)` + `LoadedAlarmIds`; that alone would have clobbered the node's severity and message, so the projection also carries the definition's severity, the resolved message template, and the last-observed worst-input quality. - Only non-normal alarms. One in the no-event position already matches what the materialise built, so a never-fired alarm stays byte-identical to a deploy without this pass (including its Message, which the materialise seeds with the display name), and an all-normal deploy writes nothing at all. - Timestamp is the condition's own `LastTransitionUtc`, not the deploy instant. - No duplicate Part 9 events: `WriteAlarmCondition`'s delta-gate compares against the node's CURRENT state, so a re-assert onto a freshly materialised node is a genuine delta (fires once — correct, the rebuild reset what clients see) while one onto a surgically-preserved node is suppressed. Tests: 6 new host tests + 6 new engine tests. Falsifiability checked by disabling the call — 4 of the 6 host tests go red; the other 2 are absence guards against an over-broad fix and pass either way by design. Live gate (docker-dev, central-2, pre-fix image vs. rebuilt image): - pre-fix: condition absent from ConditionRefresh after a rebuild; - post-fix: present, Unacknowledged, Retain=True, stamped with its own LastTransitionUtc rather than the restart instant; - /alerts stayed empty across two re-asserts, with the panel proven live by a real ACTIVATED transition immediately afterwards — no duplicate history. |
||
|
Joseph Doherty
|
28c2866710 |
merge: Sql driver follow-ups #496/#497/#498 + the Runtime.Tests flake #500 (fix/sql-driver-followups)
Four commits closing the three follow-ups the Sql poll driver left behind, plus the intermittent Runtime.Tests failure found while verifying them. - #497 corrects 16 wrong OPC UA status-code constants across 6 drivers (the issue named 3) and adds StatusCodeParityTests, a reflection guard checking every driver's hard-coded uint against the pinned SDK. That guard is what found the other 13. - #498 fails the deploy when a Sql driver's persisted config carries a literal connectionString — the write-side half of a guarantee only enforced on read. - #496 implements the design 8.1 catalog gate: authored table/column names are resolved against the live catalog at Initialize and replaced with the catalog's own spelling, so quoting becomes the backstop it was documented to be rather than the sole defence. - #500 fixes the Runtime.Tests flake: four ordering/logic defects plus a class of tight presence budgets, now routed through a documented RuntimeActorTestBase.PresenceBudget. Verified: full solution builds, all 41 unit-test projects green, Sql integration suite 21/21 against the real SQL Server on 10.100.0.35, and 30 consecutive Runtime.Tests runs clean against a measured 13% baseline. Still open by design: #499 (ClusterNode.DriverConfigOverridesJson is a third config persistence surface DraftValidator cannot see) and #501 (two alarm-ack tests use AwaitAssert for an absence assertion, so they prove nothing; the rewrite may legitimately turn them red). |
||
|
Joseph Doherty
|
154171f48c |
test(runtime): fix the intermittent Runtime.Tests failure — 4 ordering/logic defects + the presence-budget class (#500)
The reported symptom was "Runtime.Tests occasionally reports Failed: 1" with no test
name. Instrumenting 30 full-assembly runs reproduced it at 13% (4 runs) and showed it
was never ONE flaky test: five failures across three distinct tests in that batch, and
five more distinct tests over the verification rounds that followed.
Two hypotheses were tested and DISPROVED by measurement before anything was changed:
- Cluster-formation timeout. Every test in this assembly forms a real single-node Akka
cluster over TCP in RuntimeActorTestBase's constructor and waits 5 s for MemberStatus.Up
— 219 formations per run. Instrumented: idle max 960 ms, under-load max 1470 ms, zero
timeouts. A 3.4x margin; not the cause.
- Ephemeral-port exhaustion from that bind churn. TIME_WAIT measured at ~0 throughout.
Not the cause.
FOUR GENUINE ORDERING/LOGIC DEFECTS (none is a timeout):
1. VirtualTagHostActorTests.ApplyVirtualTags_respawns_child_when_plan_changes_in_place
The test loops to accept RegisterInterest/UnregisterInterest in EITHER order — and
then asserts on mux.LastSender, which DEPENDS on that order. Under the [Register,
Unregister] interleaving LastSender is the DYING child and the assertion fails. Now
captures the sender of the Register message as it arrives. Fully deterministic; no
timing involved. (Swept the other four LastSender sites: all drain the Unregister
first, so their ordering is fixed. Only this one was unsafe.)
2. DriverHostActorWriteRoutingTests.Primary_routes_write_via_uns_nodeid_to_driver_by_rawpath
ApplyAck marks the end of the APPLY, not the point a write can succeed: the child is
still in Connecting, which deliberately fast-fails writes ("driver not connected"),
and the NodeId->driver reverse map has not been pushed. Now retries until accepted.
This does not weaken the assertion — every rejection branch replies WITHOUT reaching
the driver, so Writes.Count.ShouldBe(1) still means exactly what it did.
3. HistorianAdapterActorTests.Redundancy_snapshot_without_local_node_...
One 500 ms constant served both presence budgets (AwaitAssert) and absence windows
(ExpectNoMsg). Different quantities that happened to share a number, so the presence
budget could not be raised without slowing every absence check. Split into
AssertTimeout (5 s) and Settle (500 ms).
4. ContinuousHistorizationRecorderTests.Retry_after_writer_failure_eventually_acks
Waited on `outbox == 0`, which is ALSO true before the first append — so the poll
could win the race against the recorder and return having observed the INITIAL state,
after which the CallCount>=2 check outside the block failed against a recorder that
had not run. Both conditions are now polled together with the retry count as the
discriminator. (The preceding test in the same file documents this exact trap.)
THE PRESENCE-BUDGET CLASS:
After those four, three consecutive 30-run rounds each still failed once — on a
DIFFERENT test, in a DIFFERENT file, every time (OpcUaPublishActorApplyFailureTests 2 s,
OpcUaPublishActorRebuildTests 2 s, OpcUaPublishActorTests 500 ms). That is a
distribution, not a defect, and fixing it one test at a time was the wrong method.
RuntimeActorTestBase now carries a documented PresenceBudget (15 s) and 57 sub-3 s
AwaitAssert/AwaitCondition durations across four files route through it.
Why this is safe rather than sloppy: a presence budget is an upper bound before giving
up, not a wait — these helpers poll and return the instant the condition holds. Raising
one costs nothing on the happy path and CANNOT make a genuinely failing assertion pass;
it only changes how quickly a real breakage reports. Absence windows are the opposite
(the elapsed time IS the assertion) and were deliberately left untouched with their own
short, individually-calibrated literals.
ScriptedAlarmHostActorTests is a special case, diagnosed rather than merely raised: its
8 s budget was sized for message passing but actually waits on a REAL Roslyn compile
(ApplyScriptedAlarms -> ScriptedAlarmEngine.LoadAsync compiles every predicate). Cold
script compiles are the slowest and most variable thing in the assembly, so that class
gets 30 s with the reason recorded.
Verification: 30 consecutive full-assembly runs, 30 passed / 0 failed, against a 13%
baseline. Full solution builds; all 41 unit-test projects green.
NOT fixed here, filed as #501: DriverHostActorNativeAlarmAckRoutingTests:90 and :116 use
AwaitAssert for an ABSENCE assertion, so they return instantly and prove nothing. They
are excluded from the budget change on purpose — a bigger number does not fix them, and
rewriting them to settle-then-assert may legitimately turn them red.
|
||
|
Joseph Doherty
|
4dc2ad8084 |
feat(sql): implement the design §8.1 catalog identifier-validation gate (#496)
Until now ISqlDialect.QuoteIdentifier was the SOLE defence for authored
table/column names: an identifier went from the TagConfig blob straight into a
command text, bracket-quoted. The residual risk was bounded — a hostile name is
quoted into one nonexistent object, the query fails after the connection opens,
and the tag Bad-codes — but "bounded" is not "filtered", and the design promised
a filter. Both ISqlDialect and SqlServerDialect carried a doc paragraph saying so.
SqlCatalogGate + SqlCatalogLoader now resolve every authored identifier against
the live catalog at Initialize, and REPLACE it with the catalog's own spelling.
The identifier text in an emitted poll query is therefore a string this driver
read back out of ListSchemas/ListTables/ListColumns — not operator input. Quoting
becomes the backstop it was documented to be.
Decisions worth knowing before touching this:
- Substitution, not just validation. Matching is exact-ordinal first, then a
UNIQUE case-insensitive hit: SQL Server's default collation is CI so case
variants have always worked, and rejecting them would break valid deployments.
An ambiguous CI match under a case-sensitive collation is refused rather than
guessed — picking one would publish another column's data under the operator's
node, which is worse than rejecting the tag.
- Charset check BEFORE catalog lookup. Each identifier goes through
QuoteIdentifier for its rejection rules first, so a name carrying a control or
Unicode format character is rejected WITHOUT its value being echoed into a log
line (Trojan-Source). A name that passes is safe to render, which is why
catalog-miss messages do name it — an operator hunting a typo has to see what
they wrote. Both halves are pinned by tests.
- A rejected tag keeps its node. It is dropped from the POLLED table but stays in
the AUTHORED table, so it still materializes and reads BadNodeIdUnknown —
§8.1's specified outcome. My first wiring dropped it from both, which deleted
the node instead; an existing test (ReinitializeAsync_recoversFromFaulted)
caught it. A status code can only be published by a node that exists, and a
missing address-space entry is far harder to diagnose than a Bad quality.
- An unreadable catalog FAULTS Initialize; it does not reject every tag. That is
the absence of evidence about the tags, not evidence against them — rejecting
all of them would serve a confidently-empty address space and send the operator
hunting typos that do not exist. Zero visible schemas is treated the same way,
because that is exactly what a missing GRANT looks like. Faulting lands
DriverInstanceActor in Reconnecting with its retry timer running.
- Bounded load: one schema list, one default-schema scalar, then one ListTables
per distinct authored schema and one ListColumns per distinct authored relation
— never a full catalog enumeration, and nothing after Initialize. Every
authored name reaches the catalog queries as a bound @schema/@table parameter,
so building the allow-list cannot itself be an injection vector.
- ISqlDialect gains DefaultSchemaSql ("SELECT SCHEMA_NAME()"). An unqualified
`TagValues` must resolve in whatever schema the SERVER reports; hardcoding dbo
would be a silent lie on any estate that maps service accounts to their own
default schema. It is a query, not a constant, because the answer is
per-connection.
- Accepted v1 limitation: a 3-part db.schema.table (or linked-server name)
addresses a catalog this connection cannot enumerate, so it cannot be
allow-listed and is rejected with a message pointing at the fix — expose the
data through a view in the connected database.
VerifyLivenessAsync's wall-clock pattern is extracted to RunBoundedAsync and
reused, so the new I/O gets the same R2-01 / STAB-14 protection rather than a
second hand-rolled copy. (It also fixes a latent bug in the extracted code: the
OperationCanceledException arm detached the wrong task.)
Tests: 24 pure gate tests + 9 end-to-end driver tests against the real SQLite
catalog + 3 new live tests against the real SQL Server on 10.100.0.35 (21 in that
suite now pass, exercising the real SELECT SCHEMA_NAME() + INFORMATION_SCHEMA
path). Verified falsifiable: bypassing the gate turns exactly the three rejection
assertions red and leaves the rest green.
SqlInjectionRegressionTests is deliberately NOT rewritten to expect
BadNodeIdUnknown. It drives SqlPollReader directly, below the gate, and pins the
quoting backstop on its own — defence in depth is only worth the name if each
layer holds independently. Rewriting those assertions would delete the backstop's
only coverage and leave the gate a single point of failure. Its scope note, which
said the gate does not exist, is updated to say why it stays where it is.
|
||
|
Joseph Doherty
|
1919a8e538 |
feat(config): reject a persisted Sql connectionString at the deploy gate (#498)
The Sql driver's "a pasted literal connection string is never persisted" guarantee was
enforced only on the READ path: SqlDriverConfigDto has no connectionString property and
UnmappedMemberHandling.Skip drops the key on deserialization. Nothing stopped it being
WRITTEN. Config blobs are schemaless JSON columns and the fallback authoring pattern for
a driver without a typed form is a raw-JSON textarea, so an operator pasting
{"connectionString":"Server=...;Password=..."} would put a live credential in the
ConfigDb — persisted, replicated to every node's artifact cache, readable by anyone with
config access — while the runtime silently ignored it and the driver failed to connect.
Discarding a secret on read is not the same as refusing to store it.
DraftValidator.ValidateSqlConnectionStringNotPersisted fails the deploy
(SqlConnectionStringPersisted) when a Sql-typed driver instance's DriverConfig, or the
DeviceConfig of any device beneath it, carries a top-level connectionString key.
- Both surfaces are checked because DeviceConfig is merged onto DriverConfig before the
DTO sees it — a credential pasted into the device blob is the identical leak.
- The key is matched case-insensitively: System.Text.Json binds ConnectionString to a
connectionString property by default, so an ordinal match would leave the obvious
bypass open.
- Only the top level is scanned; the DTO is flat, so a nested occurrence cannot bind and
is not the mistake this rule exists to catch.
- The message never echoes the value — validation errors reach the AdminUI, the deploy
log and the audit trail, and repeating the string there would leak the credential the
rule is refusing to store. Pinned by a test.
- Malformed/blank/non-object JSON never throws: a parse failure would take down every
other rule in the same pass.
16 tests. Verified falsifiable — with the rule disabled the 6 positive assertions fail
and the 10 negatives stay green, so none of them passes vacuously.
Design §8.2's "if an inline connectionString is ever permitted (dev convenience only),
the AdminUI redacts it" carve-out is withdrawn in the same change: redaction only hides a
credential that has already been written.
Not covered, and filed as a follow-up: ClusterNode.DriverConfigOverridesJson is a third
persistence surface for a driver config blob, but it is not part of DraftSnapshot, so
this validator cannot see it.
|
||
|
Joseph Doherty
|
b95b99ba8e |
fix(drivers): correct 16 wrong OPC UA status-code constants + guard them by reflection (#497)
Every value verified against Opc.Ua.StatusCodes in the pinned SDK (1.5.378.106) by reflection, not by copying siblings. All are client-visible: OPC UA clients branch on status. The three named in #497: - Historian.Gateway SampleMapper "BadNoData" 0x800E0000 -> 0x809B0000 (0x800E0000 is BadServerHalted) - Galaxy "BadTimeout" 0x800B0000 -> 0x800A0000 (0x800B0000 is BadServiceUnsupported) - TwinCAT BadTypeMismatch 0x80730000 -> 0x80740000 (0x80730000 is BadWriteNotSupported) BadTypeMismatch was wrong in three MORE drivers the issue did not name — FOCAS, AbLegacy and AbCip carried the same 0x80730000. Every call site is a FormatException/InvalidCastException conversion failure, so the name was right and the value was wrong in all four. Adding the reflection guard then surfaced nine further defects offline tests had never checked: - Galaxy GoodLocalOverride 0x00D80000 -> 0x00960000 (not a UA code at all) - Galaxy UncertainLastUsableValue 0x40A40000 -> 0x40900000 - Galaxy UncertainSensorNotAccurate 0x408D0000 -> 0x40930000 - Galaxy UncertainEngineeringUnitsExceeded 0x408E0000 -> 0x40940000 - Galaxy UncertainSubNormal 0x408F0000 -> 0x40950000 - TwinCAT BadOutOfService 0x80BE0000 -> 0x808D0000 (was BadProtocolVersionUnsupported) - TwinCAT BadInvalidState 0x80350000 -> 0x80AF0000 (was BadAttributeIdInvalid) - AbCip/AbLegacy GoodMoreData 0x00A70000 -> 0x00A60000 (was GoodCommunicationEvent) - Historian.Gateway GatewayQualityMapper Good_LocalOverride 0x00D80000 -> 0x00960000 Galaxy's whole Uncertain block read as though the OPC DA quality byte could be shifted into the UA substatus position; it cannot — the substatuses are an unrelated enumeration. GatewayQualityMapper already held the correct table, which is what the Galaxy values are now reconciled against. Guard: StatusCodeParityTests (Core.Abstractions.Tests, which already project- references every driver) reflects over every `const uint` in the deployed ZB.MOM.WW.OtOpcUa.Driver.*.dll whose name reads as a status code, and asserts it equals Opc.Ua.StatusCodes.<name>. Discovery is by convention, so a new driver assembly referenced by that project is covered with no edit here. It went red on all nine defects above before the fixes and now checks 109 constants green. Because reflection can only see a NAMED constant, two inline literals were hoisted so the guard can reach them: Galaxy's BadTimeout/Bad into StatusCodeMap, and GatewayQualityMapper's 15-entry table into a new HistorianStatusCodes. An inline `0x800B0000u, // BadTimeout` at a call site is exactly how the Galaxy defect survived. The drivers stay SDK-free by design; only the test project references Opc.Ua.Core, as the oracle. Also corrected: tests that pinned the wrong values (Galaxy StatusCodeMapTests, SampleMapperTests, GatewayQualityMapperTests — all written from the same bad source), a mislabelled comment in DriverInstanceActorTests, and stale constants in two design docs, one of them the unexecuted MTConnect driver design that would have propagated BadNoData's wrong value into a new driver. The CLI's SnapshotFormatter value->name table was checked against the SDK and is correct; no change needed. |
||
|
Joseph Doherty
|
ee12568cab |
refactor(modbus-rtu): distinct CrcMismatch/UnitMismatch desync reasons
The RTU framer overloaded DesyncReason.TruncatedFrame for three distinct failures — a genuine short read, a CRC-16 validation failure, and a CRC-valid reply from the wrong RS-485 slave. Split the latter two into dedicated CrcMismatch / UnitMismatch enum members so a future .Reason consumer (metrics, operator diagnostics) can tell a wiring/EMI CRC fault apart from a mis-addressed multi-drop reply. Behaviour is unchanged — all three still throw ModbusTransportDesyncException and tear the socket down. Framing tests now assert the specific Reason on each path. Follow-up #12 from the Modbus RTU-over-TCP plan. Claude-Session: https://claude.ai/code/session_01GASWkNEi68FSCtvr6rLoEW |
||
|
Joseph Doherty
|
d5bd4226ee |
docs(mqtt): Sparkplug P2 live-verified; MQTT driver complete
Task 26 — the P2 milestone gate. Ran the live /run verification on an isolated
docker-dev rig (project otopcua-mqtt, ports 9210/4850-4851/14350, image built
from this branch) against the real Mosquitto TLS+auth broker and the C#
Sparkplug edge-node simulator on 10.100.0.35, and recorded the result.
The gate found three defects. Two are the same defect class the P1 gate found
twice — a hand-maintained AdminUI surface left behind by a driver-side feature —
and the third is a pre-existing cross-driver bug that only a Sparkplug flow
could surface.
1. CRITICAL, FIXED — MqttDriverForm still shipped its P1 Sparkplug PLACEHOLDER.
Switching Mode to SparkplugB rendered a "not available yet" notice and NO
Group ID field, so with Sparkplug ingest fully shipped there was still no
way to author a Sparkplug driver from the AdminUI at all. Sparkplug.GroupId
is the driver's entire subscription filter (spBv1.0/{GroupId}/#): blank ⇒
connected, Healthy, ingesting nothing. Now authors all five Sparkplug keys,
MERGES over the existing sub-object rather than replacing it (so a key a
newer driver adds inside it survives an older AdminUI), leaves it untouched
in Plain mode, and validates the group id as the topic segment it is.
Pinned by 8 MqttDriverFormModelTests cases, verified falsifiable — 8 RED
with the fix stubbed out.
2. PRE-EXISTING + CROSS-DRIVER, FIXED — every node label in the shared
DriverBrowseTree was an <a href="#">. In a Blazor Web App, blazor.web.js's
enhanced-navigation click interceptor resolves a bare "#" against
<base href="/">, so clicking ANY browse-tree node label navigated the whole
AdminUI to "/", tore down the circuit, and destroyed the hosting modal —
losing the browse session AND the tag selection. @onclick:preventDefault
does not help: it suppresses the browser's default action, not Blazor's own
interceptor. Labels are now <button type="button">. Dates to the component's
introduction (2026-05-28) and affects EVERY driver's picker; it surfaced
only now because choosing a rebirth scope is the first flow that requires
clicking a label rather than the ▶ toggle (always a button, always fine).
3. FIXED (mitigation) — the browse tree rendered exactly once, at open, and
never again, while OpenAsync returns as soon as the SUBSCRIBE is granted.
On Sparkplug that means it is almost always empty forever, because births
are never retained. Added a Refresh button that re-reads the root against
the SAME session (nothing reconnects, nothing observed is lost, the tag
selection is kept, only the armed rebirth scope is cleared).
Live gate result (all against the real broker + simulator, group OtOpcUaSim):
- driver authored end-to-end through the fixed MqttDriverForm; blob is
Mode:SparkplugB + Sparkplug.GroupId:OtOpcUaSim with enums as names
- browse: BROWSER OPEN chip, Sparkplug-only rebirth panel, tree renders
Group → EdgeNode → [Device] → Metric with the device folder Filler1 and
node-level metric leaves SIDE BY SIDE, and "Node Control/Rebirth" as ONE
leaf (the metric-name-is-not-a-topic-segment rule holding)
- rebirth: group scope names the group + the 32-node cap, Cancel published
NOTHING (simulator log unchanged); edge-node scope → "1 command published"
and the simulator logged "rebirth NCMD #1 received; republishing metadata";
group scope → "2 commands published", both edge nodes re-birthed; a device
and a metric both resolve UP to "edge node EdgeA"; the panel does NOT render
for a Plain MQTT device
- browse-commit wrote bindable tuples with no topic/address key —
{"groupId","edgeNodeId","metricName","deviceId"} for the device metric and
deviceId ABSENT (not blank) for the node-level one, datatypes inherited from
the birth (Int64 / Float)
- deployed (Sealed), and both nodes served live changing values through OPC UA
at the 2 s cadence — Good 0x00000000, no BadNodeIdUnknown
- death→STALE: stopping the simulator drove both nodes to 0x80000000 with an
empty value ~2 s later, and restarting it recovered them to Good on the new
birth (FillCount back to its birth-declared 1000, then counting)
- rediscovery fired exactly TWICE (once per authored scope: OtOpcUaSim/EdgeA
7 metrics, OtOpcUaSim/EdgeA/Filler1 3 metrics) and was then gated across
many subsequent births/rebirths — the anti-storm change gate working, and
EdgeB correctly filtered out as an unauthored scope
- tag editor: opens in SparkplugB (mode inference on reopen) with all four
descriptor fields populated; blank group → "A Sparkplug group ID is
required."; "Plant/1" rejected; "Node Control/Rebirth" ACCEPTED and saved
with the slash whole; dataType override persists and its key is ABSENT again
after selecting "(from birth certificate)"
Two things are NOT verified, and are recorded rather than claimed:
- Rediscovery is INERT in v3 and this is a platform gap, not an MQTT one:
nothing subscribes to IRediscoverable.OnRediscoveryNeeded and
DriverHostActor.HandleDiscoveredNodes hard-returns. A DBIRTH introducing a
metric does NOT change the OPC UA tree; redeploy. Verified from the driver's
log line only, exactly as far as it can be verified.
- A metric name containing "/" cannot be BROWSE-COMMITTED: the derived raw tag
Name is a RawPath segment and may not contain "/", so the spec-mandatory
"Node Control/Rebirth" is refused ("Row 3: Name must not contain '/'"). The
refusal is loud and all-or-nothing, never a silently mis-bound tag, and
Manual entry is the working path. Fixing it needs a name-sanitisation policy
with a collision answer, so it was recorded rather than guessed at.
Also left open + documented: an empty browse tree cannot arm a rebirth (the
scope comes from a tree click, and the session side already accepts a bare
{group}/{edgeNode} — only a UI path is missing); _canRebirth is captured at
browse-open; the tag editor injects the Plain-only payloadFormat key into a
Sparkplug blob (cosmetic — the factory routes on the tuple).
Suites: offline 1528 passed / 0 failed (581 Driver.Mqtt + 809 AdminUI + 138
Core.Abstractions); live 15/15 against the broker + simulator.
Docs: docs/drivers/Mqtt.md brought fully up to date for P2 (Sparkplug config
sub-object, both tag shapes, the ingest state machine, rediscovery, browse +
rebirth + Refresh, a Known-gaps table); Mqtt-Test-Fixture.md gains the simulator
and drops its "Sparkplug has no fixture" claims; infra/README.md §3 gains the
simulator row; CLAUDE.md gains the simulator endpoint facts; the tracking doc
marks MQTT/Sparkplug COMPLETE. docker-dev/docker-compose.mqtt.yml is the
isolated-rig overlay the gate ran on.
Claude-Session: https://claude.ai/code/session_01GASWkNEi68FSCtvr6rLoEW
|
||
|
Joseph Doherty
|
8d9155682d |
fix(mqtt): browse-commit writes a bindable address + a Request-rebirth affordance
Two gaps the Task 23/24 review found, both blocking Task 26's live gate.
Gap 1 — browse-commit produced a silently dead MQTT tag. RawBrowseCommitMapper
had no `Mqtt` case, so a committed leaf fell through to the generic
`{"address": …}` key. Neither MqttTagDefinitionFactory entry point reads
`address`: the tag deployed clean and reported BadNodeIdUnknown forever, with
no signal at commit time. Predates Task 23 (Plain was affected too), but Task 23
built the Sparkplug metric tree precisely so an operator could browse and commit
a binding.
The address is a DESCRIPTOR, not a reference string, and it cannot be recovered
from the browse node id: `{group}/{node}[/{device}]::{metric}` where a metric
name legitimately contains `/` (`Node Control/Rebirth`) — the ambiguity
MetricSeparator's remarks already name. So the session STATES it, via a new
`AttributeInfo.AddressFields` seam, and the mapper reads it. Which keys are
emitted is also how the mapper learns Plain vs Sparkplug — the driver type
reaching it is just `Mqtt`, and the mode lives on the driver config. Key names
are single-sourced in the new `MqttTagConfigKeys` (producer, mapper, factory),
with the literals pinned by a test so a symmetric rename cannot silently unbind
already-persisted blobs. A leaf with no stated address is refused at commit in
words rather than committed dead.
Gap 2 — RequestRebirthAsync had no UI. Task 23 shipped it backend-only; Task 26
step 1 assumes the button, and Task 25's runbook notes the picker tree stays
empty until a birth lands, so it is the only way to fill it on demand. Added to
the /raw browse modal: scope = the clicked tree node (device/metric resolve up to
their edge node, group fans out and is refused whole past 32), an explicit
two-click confirm naming the resolved scope and its consequence, the outcome or
the refusal shown rather than swallowed. Offered only for a Sparkplug session
(new `IRebirthCapableBrowseSession.RebirthAvailable` — one session class serves
both modes, so a type test alone would draw a button that can only throw) and
only to a DriverOperator; the server-side gate remains the boundary.
Tests: MQTT 545 → 581, AdminUI 781 → 800. The round-trip tests feed the emitted
blob to the REAL factory and were falsified by mutating the emitted key name.
Claude-Session: https://claude.ai/code/session_01GASWkNEi68FSCtvr6rLoEW
|
||
|
Joseph Doherty
|
767e7031b3 |
test(mqtt): C# Sparkplug edge-node simulator + §3.6 live matrix
Adds a project-owned, controllable Sparkplug B edge node (SparkplugEdgeNode) that
encodes with the SAME generated Tahu schema the driver decodes with — so the live
gate checks encode/decode symmetry rather than the driver against itself — and
drives the §3.6 matrix end-to-end over the real TLS+auth Mosquitto fixture.
The simulator frames its topics independently of SparkplugTopic.Format: a
simulator borrowing the parser's formatter could not detect a bug in it, because
both sides of the comparison would be wrong together. It registers a real NDEATH
Last Will at CONNECT, restarts seq at 0 on NBIRTH, publishes DATA metrics as
alias-only, carries bdSeq per session, and encodes signed ints as two's
complement in the unsigned proto field.
SparkplugLiveTests (Category=LiveIntegration, env-gated, skip-clean) covers:
birth -> alias-only data -> OnDataChange under the RawPath; late-join rebirth
NCMD honoured by an independent decoder; alias reuse across a rebirth routing by
metric NAME; a stale-bdSeq NDEATH ignored while the current one stales; the real
broker-published Will staling and the next birth restoring; seq wrap 255->0
requesting no rebirth while a deliberate gap does; and the browser's passive
window asserted ON THE WIRE (a third client watching spBv1.0/{group}/NCMD/#),
plus RequestRebirthAsync node-vs-group enumeration.
Two things make the suite falsifiable rather than decorative: the seq-wrap test
builds its ingestor with rebirthDebounce: TimeSpan.Zero (at the shipped 10s
default a wrap-triggered NCMD would be swallowed and the test would pass for the
wrong reason) and pairs the silence with a deliberate gap that must produce one;
and the passivity assertion observes the broker rather than the session's own
publish counter, which can only see the seam it guards.
Docker/docker-compose.yml gains a profile-gated `sparkplug-sim` service running
the same engine standalone against the same broker (TLS, CA-pinned) — a manual
driving aid for the AdminUI picker, deliberately NOT a test dependency, since the
matrix needs an edge node it can command mid-test. Its app directory is publish
output (publish-simulator.sh), gitignored like secrets/.
Offline: 15 skipped, 0 failed with no env set. Live: 15/15 passed against
10.100.0.35:8883. Falsifiability spot-check: mutating AliasTable.RebuildFromBirth
to merge instead of replace, and neutering SparkplugCodec.ReinterpretSigned for
Int32, turned exactly the two corresponding tests red and nothing else; both
mutations reverted.
Claude-Session: https://claude.ai/code/session_01GASWkNEi68FSCtvr6rLoEW
|
||
|
Joseph Doherty
|
64ec7aa43a |
feat(mqtt): Sparkplug birth-driven browser tree + scoped Request-rebirth action
Unseals the Sparkplug branch of the MQTT address-picker browser that Task 10 deliberately sealed shut, and adds the first (and only) sanctioned publish on a browse session. - MqttBrowseSession serves Group -> EdgeNode -> [Device] -> Metric in Sparkplug mode, decoded from observed NBIRTH/DBIRTH certificates (the only Sparkplug messages that name their metrics). Node-level metrics hang directly under their edge node -- no synthesised device folder, because the authored address tuple genuinely has deviceId = null for them. Metric node ids use a '::' separator: a metric name may contain '/', so slash-joining would make a node-level metric indistinguishable from a device folder. - AttributesAsync is purely birth-derived in Sparkplug mode. The plain path's UTF-8 inference / payload-snippet machinery does not run and no payload bytes are retained: a Sparkplug body is protobuf and would be reported as "(N bytes, binary)" for every metric in the plant. A datatype ToDriverDataType cannot map is reported as the Sparkplug type name, never coerced. - RequestRebirthAsync(scope) is the one publishing member, reached only by an explicit operator action. It routes through the counted PublishAsync chokepoint via a private RebirthTransport adapter, so PublishCountForTest still proves that OpenAsync/RootAsync/ExpandAsync/AttributesAsync/Observe/ DisposeAsync publish nothing -- now in both modes. A device or metric scope resolves up to its owning edge node (NCMD is node-addressed); group scope enumerates observed edge nodes and is refused whole past MaxGroupRebirthNodes = 32. - BrowserSessionService.RequestRebirthAsync gates on the same DriverOperator policy that gates the picker, evaluated server-side where the action happens rather than only at render time, fails closed, and Info-logs the scope and the published count. Exposed through the new IRebirthCapableBrowseSession contract so "does this session write?" stays a type question. - ToBrowseOptions' Last-Will landmine re-verified: MqttDriverOptions still carries nothing will-shaped (an NDEATH is a broker-published will and would be invisible to PublishCountForTest), now pinned by a reflection guard. Falsifiability: injecting a publish into RootAsync reddens the Sparkplug and plain passivity tests; double-publishing per target reddens all four one-NCMD-per-node assertions. 572/572 MQTT tests, 24/24 AdminUI browsing tests. Claude-Session: https://claude.ai/code/session_01GASWkNEi68FSCtvr6rLoEW |
||
|
Joseph Doherty
|
6d7a458c4d |
fix(mqtt): reset Sparkplug ingest state at every session/authoring seam
Task 21 review — two Criticals sharing one root cause, plus four follow-ups.
C1/C2 root cause: `OnReconnectedAsync` opened with `Births.Clear(); _trackers.Clear();`
— "not housekeeping, the correctness step" — and that reset was missing from the
other two seams where the view underneath a surviving cache changes.
- C1: the ingestor OUTLIVES a session rebuild. `SessionIdentity` is a strict
superset of `IngestIdentity`, so changing only Host/Port/ClientId/credentials/
TLS/a timeout gives `!SameSession && SameIngest` — `ReinitializeAsync` tears the
session down and re-establishes on the same instance, and `TeardownAsync` touches
no ingest state (the resilience layer re-running `InitializeAsync` is the second
path). A node that rebound alias 5 during the changeover would publish Pressure's
value under the Temperature RawPath at Good. Hoisted into `ResetSessionState`,
called from `AttachTo` (earliest seam — a persistent session can push between
CONNACK and our SUBSCRIBE), `EstablishAsync` and `OnReconnectedAsync`.
- C2: `Register` pruned trackers but not births. While a node is unauthored its
traffic is dropped at `Dispatch`, so its cached birth FREEZES rather than
refreshing; drop-then-re-add across a week mis-routes at Good, and `Births` grew
monotonically. Now evicted on edge-node departure.
Deliberately NARROWER than "absent from ByScope": a device with no authored tags
under a still-authored node keeps receiving traffic, so its birth is live, not
frozen — evicting it would discard correct state. Pinned both ways.
- I1: a birth whose seq is unreadable produced a permanent debounce-paced rebirth
loop. "A birth arrived" is now tracked separately from "the birth established a
baseline". Writing the test showed the cap belonged wider than the review scoped
it: data-before-birth and unknown-alias loop identically (20 messages → 23 NCMDs
against a cap of 3), so all three missing-metadata reasons share one per-node
consecutive cap, re-armed by any applied birth. A seq gap stays uncapped — it
self-limits.
- I2: the oversize `WarnOnce` key was the raw topic, off a group-wide `#`
subscription. `HandleMessage` now parses and filters to authored nodes BEFORE the
size check (also skipping the protobuf parse for discarded traffic), and `_warned`
carries a hard ceiling so a future bad derivation degrades to silence.
- I3: array metrics slipped the unsupported-type gate — `ToDriverDataType` returns
the ELEMENT type, so a String-typed tag published a packed `byte[]` as base64 at
Good. Gated on `IsSparkplugArray`.
- I4: driver-level Sparkplug coverage for the composition-root lines T22 left
untested — the `OnDataChange` re-raise, `ReadAsync` off the shared cache,
subscribe/unsubscribe dispatch, and both directions of the mode gate.
Minors: M1 the oversize test fed unparseable bytes and passed with the size check
deleted (now a real NBIRTH + an at-the-ceiling control); M2 answered in-place (an
absent seq is refused WITHOUT becoming the baseline, so the NCMD assertion is the
complete check); M3 `HostStateObserved` raise wrapped; M5 the legacy `STATE/{host}`
form is now subscribed, so the parser's tolerance is reachable in production; M8
`ResolveBinding` prefers the NAME over a disagreeing alias; M9 a narrowing float
conversion refuses instead of publishing ±Infinity at Good (a publisher's own
infinity still passes through). M4 was already resolved by T22.
Also fixed a race in the new tests: `publish.Topics.Clear()` after a call that
dispatches rebirths off-thread must drain first — it made the C1 sequence-baseline
pin pass vacuously in the first falsifiability run.
Falsifiability: dropping the reset from AttachTo+EstablishAsync reddens 4 (C1);
dropping the Register eviction reddens 2 (C2); widening the eviction predicate to
the reviewer's literal phrasing reddens the keep-live-births pin (C2b). 545/545 MQTT
unit tests green (was 524); whole-solution `--no-incremental` build clean under TWAE.
Claude-Session: https://claude.ai/code/session_01GASWkNEi68FSCtvr6rLoEW
|
||
|
Joseph Doherty
|
cd0157a3b8 |
feat(mqtt): typed tag editor Sparkplug mode + validation
Task 24 (P2): replaces the P1 Sparkplug Validate() stub (always null) and the "not available yet" editor placeholder with real groupId/edgeNodeId/deviceId/ metricName authoring, mirroring MqttTagDefinitionFactory.FromSparkplugTagConfig in both directions -- required ids + optional deviceId, dataType/qos read strictly but only when present (dataType is optional: the birth certificate declares it). One rule is deliberately stricter than the factory: group/edge- node/device ids reject '/', '+', '#' (topic-segment characters a decoded incoming id can never carry, so an authored one that does can never bind); metricName is exempt since Sparkplug's own canonical names use '/' (e.g. "Node Control/Rebirth"). No FullName key is written -- TagConfig carries no identity under v3, and the plan's snippet asking for one was corrected per the brief. A SparkplugB tag now survives save/reopen because its descriptor keys re-infer Mode on load; there was never a 'mode' key to persist. Claude-Session: https://claude.ai/code/session_01GASWkNEi68FSCtvr6rLoEW |
||
|
Joseph Doherty
|
6917d8805d |
feat(mqtt): Sparkplug UntilStable discovery + rediscover-on-DBIRTH
RediscoverPolicy is now mode-dependent: UntilStable in Sparkplug B (a tag's dataType is optional -- the birth certificate declares it, so the discovered tree fills in asynchronously), Once in Plain (nothing about the authored set arrives on the wire). DiscoverAsync resolves each Sparkplug tag's datatype per pass from the live BirthCache, with the ingest path's own precedence: authored dataType wins, else the birth's, else the record default. An unsupported Sparkplug type (DataSet/Template/PropertySet/Unknown) falls back rather than blanking the tag. The authored tag SET is never conditional on a birth -- an authored tag is part of the declared configuration, not something the plant grants by publishing. SparkplugIngestor.BirthObserved is wired to RaiseRediscoveryNeeded behind a two-part change gate, which is the anti-storm mechanism rather than an optimisation: fire only for a scope an authored tag binds, and only when the birth's metric-name SET (ordered-distinct, ordinal) differs from the last one seen for that scope. Edge nodes re-birth freely -- on their own timer, on every reconnect via the late-join rebirth fan-out, and once per rebirth NCMD the gap policy sends -- and firing on each of those would make a healthy plant a permanent address-space-rebuild loop. No extra debounce: what survives the gate is a real edit to the served tree, and delaying it would need its own trailing-edge flush to avoid dropping the last change. The signature map is NOT cleared on death/reconnect/ingest rebuild (the ingestor drops its whole birth cache on reconnect, but "the driver forgot" is not "the address space changed"); it is pruned when a redeploy stops authoring a scope. ScopeHint is the "Mqtt" discovery folder, deliberately not the Sparkplug scope path the plan named: the discovered tree is flat, so an edge-node/device path would name a subtree that does not exist and a consumer scoping a rebuild on it would rebuild nothing. The scope is carried in Reason instead. Falsifiability: always-fire reddens the identical-rebirth + reordered-rebirth pins; always-authored-datatype reddens the birth-fill pin; dropping the authored-scope filter reddens the unauthored-device pin. 524/524 MQTT unit tests green (was 513). Claude-Session: https://claude.ai/code/session_01GASWkNEi68FSCtvr6rLoEW |
||
|
Joseph Doherty
|
8538fb798b |
feat(mqtt): Sparkplug ingest state machine (birth/alias/seq/rebirth/death→stale)
Task 21 — the design doc's §3.6 correctness core. `SparkplugIngestor` owns the
authored `(group, node, device?, metric) → RawPath` index, the live `BirthCache`,
one `SequenceTracker` per authored edge node, and the rebirth policy; it routes
NBIRTH/DBIRTH/NDATA/DDATA/NDEATH/DDEATH/STATE into `OnDataChange` +
`LastValueCache`, or into a bounded, per-node-debounced rebirth NCMD.
Invariants held (and each pinned by a falsifiability control):
- Published reference is the RawPath, never a composed Sparkplug reference —
`DriverHostActor`'s dual-namespace fan-out is RawPath-keyed.
- Tags bind by stable metric NAME; the alias is only the per-birth lookup, so an
alias reused across a rebirth cannot mis-route.
- Every value goes through `SparkplugMetricBinding.Reinterpret` before publish.
- NDEATH never reaches `SequenceTracker.Accept` (it carries no seq); it is tied to
its birth by bdSeq instead.
- An invalid payload mutates nothing — no tracker, no alias table, no eviction.
- `HandleMessage` never throws on MQTTnet's dispatcher thread.
Supporting changes:
- `MqttTagDefinitionFactory.FromSparkplugTagConfig` — the driver had no way to
parse the Sparkplug descriptor keys at all; the P1 stub fields on
`MqttTagDefinition` were never populated. Mode selects the parser, never a
blob heuristic. `dataType` becomes optional in the Sparkplug shape (the birth
declares it), recorded via the new `DataTypeAuthored` flag.
- `MqttConnection` implements `IMqttPublishTransport` (bounded, refusal throws).
- `MqttDriver` dispatches every capability by mode, subscribes `spBv1.0/{group}/#`
(+ the configured STATE topic) once at connect, and re-subscribes + requests a
late-join rebirth on reconnect. `IngestIdentity` now names `MqttSparkplugOptions`,
closing the silent same-ingest gap its own ⚠️ comment warned about.
Primary-host STATE *publishing* is explicitly out of scope and warned about at
construction: it needs the retained-OFFLINE Last Will set at CONNECT time, and
shipping the ONLINE half alone is worse than shipping neither.
58 new tests (MQTT suite 455 → 513, all green).
Claude-Session: https://claude.ai/code/session_01GASWkNEi68FSCtvr6rLoEW
|
||
|
Joseph Doherty
|
31a98a1bf5 |
feat(mqtt): RebirthRequester NCMD encode
Claude-Session: https://claude.ai/code/session_01GASWkNEi68FSCtvr6rLoEW |
||
|
Joseph Doherty
|
3fdf24659f |
fix(mqtt): AcceptBirth is NBIRTH-only -- a DBIRTH routed there wipes bdSeq
Self-review of the commit before this one caught a defect in the contract it published rather than in the code it ran. AcceptBirth's doc said it records "a birth (NBIRTH/DBIRTH)". That is wrong on both halves of this type, and the wrong version plants exactly the defect the bdSeq tie exists to prevent. Only the NBIRTH restarts the Sparkplug sequence; a DBIRTH carries the next number in the edge node's ongoing stream, so it belongs to Accept. And only the NBIRTH and NDEATH carry bdSeq at all -- a DBIRTH has none to pass. So a Task 21 that followed the old doc and routed a DBIRTH to AcceptBirth would do two wrong things at once: rebase the sequence mid-stream, and wipe the node's session token to null. A stale Last Will arriving afterwards would then find nothing to compare against, fall through the deliberate fail-toward-stale rule in IsDeathForCurrentSession, and kill the live node. Renamed AcceptBirth -> AcceptNodeBirth so the DBIRTH call site reads wrong at a glance rather than only in prose, documented the hazard on both methods, and added DeviceBirth_ContinuesTheNodeSequence_AndLeavesTheSessionTokenAlone to pin the shape at this type's own boundary -- the ingest state machine now has something to fail against if it wires the call sites the other way round. Falsifiability: mutating Accept to clear _birthBdSeq reddens exactly the new test (1 RED), reverted. 29 tests, 438/438 for the MQTT suite. Claude-Session: https://claude.ai/code/session_01GASWkNEi68FSCtvr6rLoEW |
||
|
Joseph Doherty
|
e4e313cf1c |
feat(mqtt): SequenceTracker seq-gap + bdSeq death-tie
SequenceTracker (Driver.Mqtt/Sparkplug/) holds one edge node's Sparkplug stream-continuity state -- design doc SS3.6 invariant #3. Two independent halves: the wrapping 0-255 seq counter that reports whether a message was missed, and the bdSeq session token that ties an NDEATH to the NBIRTH it belongs to. Next-expected is (last + 1) & 0xFF, so 255 -> 0 is the sequence continuing. The boundary is the whole task because it fails differently in each direction: read the wrap as a gap and every edge node is asked to rebirth once per 256 messages, so a busy node spends its life republishing births instead of data; miss a real gap and the driver serves values it knows are behind the device, at Good quality, indefinitely. Both directions are asserted, plus two full wrap cycles so an off-by-one that only misfires at one value has nowhere to hide. A gap is reported once and the tracker then resynchronizes onto the observed value -- holding the baseline at the pre-gap number would make every following message report a gap too, so one lost message would become a permanent rebirth storm, the same pathology as a wrong wrap reached by a different route. An out-of-range seq is refused, never masked. The codec hands seq over as a ulong precisely so a spec-violating publisher's value is not truncated into a plausible-looking one, and masking would finish the job it declined to do: with the baseline at 255, a seq of 256 masks to 0 -- exactly what the wrap expects next -- and the bogus message would be accepted as contiguous. Refused values do not become the baseline either, so the next genuine message is still measured against the last credible one. A birth restarts the sequence via AcceptBirth rather than Accept, so it is never itself a gap; a gap after a birth is still detected. An in-range but non-zero birth seq is adopted rather than refused -- refusing it would demand a fresh rebirth for every message from an otherwise-followable publisher. The first message on a virgin tracker cannot be a gap (nothing to measure it against) but leaves IsBirthSynchronized false, which is how Task 21 tells a mid-stream late join from a synchronized node. bdSeq exists to stop a late Last Will from killing a live node: a node's connection drops, it reconnects and publishes a fresh NBIRTH, and only then does the broker notice the old connection died and deliver its will carrying the PREVIOUS session's bdSeq. Without the compare that stale NDEATH drives every tag of a freshly-born healthy node to Bad. Unknowns fail toward stale deliberately -- only a positive mismatch of two known tokens discards a death, because acting on a death wrongly self-corrects at the next birth (invariant #4) while ignoring a real one leaves a dead node's values flowing Good forever. bdSeq is not a top-level field, so TryReadBdSeq extracts it from the metric named bdSeq and runs it through ReinterpretSigned first: read raw, a negative Int64 bdSeq becomes an enormous ulong -- a plausible-looking session token minted out of nonsense. Falsifiability verified, each mutation reverted after observing RED: (a) no-wrap expected -> 2 RED; (b) Accept always true -> 5 RED; (c) bdSeq compare always matches -> 2 RED; (d) mask out-of-range seq into range -> 2 RED. 28 tests, 437/437 for the MQTT suite. Claude-Session: https://claude.ai/code/session_01GASWkNEi68FSCtvr6rLoEW |
||
|
Joseph Doherty
|
2afef00eaa |
feat(mqtt): AliasTable/BirthCache — bind-by-name, rebuild-per-birth
Design §3.6 invariants #1/#2. A Sparkplug alias is scoped to ONE birth: after a rebirth an edge node may point alias 5 at a different metric. So authored tags bind by the stable metric NAME and the alias is a per-birth cache only — RebuildFromBirth builds both indexes fresh and publishes them in one assignment, never merging or upserting, including for an empty birth (which legitimately clears the scope). BirthCache keys scopes by the full (group, edgeNode, device?) triple and applies the spec's node→device rules: an NBIRTH invalidates every DBIRTH under that node (returning their metric sets for the STALE fan-out), a death evicts rather than blanks so data-before-rebirth stays detectable. Reads are lock-free on MQTTnet's dispatcher thread — an immutable snapshot swapped atomically per scope, the same discipline MqttSubscriptionManager.AuthoredTable uses, with rebuilds landing in place so a held table reference always observes the newest birth. Running values are deliberately NOT stored here — LastValueCache owns those, keyed by RawPath. SparkplugMetricBinding.Reinterpret exposes the birth datatype to the codec's two's-complement fix, since a DATA metric carries no datatype. Falsifiability: mutating RebuildFromBirth to merge reddened 5/19 tests (both plan invariants); making the alias carry metric identity across a birth reddened 3/19, including the headline alias-reuse case. Claude-Session: https://claude.ai/code/session_01GASWkNEi68FSCtvr6rLoEW |
||
|
Joseph Doherty
|
b245f380b1 |
feat(mqtt): Sparkplug topic parse/format + datatype map
SparkplugTopic (Driver.Mqtt/Sparkplug/) parses/formats spBv1.0 topics for
every message type (NBIRTH/DBIRTH/NDATA/DDATA/NDEATH/DDEATH/NCMD/DCMD/STATE).
TryParse never throws -- it is fed every topic a live spBv1.0/{group}/#
subscription delivers -- and rejects device/node-scope mismatches and MQTT
wildcard chars in a segment. STATE is handled honestly rather than force-fit
into the {group}/{type}/{node} mould: it parses both the v3.0
spBv1.0/STATE/{hostId} form and the legacy no-prefix STATE/{hostId} form
(tolerated on receive only -- Format/FormatState always emit v3.0). Format
gives Task 20's NCMD/DCMD write path a builder instead of hand-concatenation.
SparkplugDataType is a `global using` alias for the vendored proto's
generated Org.Eclipse.Tahu.Protobuf.DataType, not a second hand-duplicated
enum -- Metric.Datatype is a raw wire uint32 (no enum-typed field forces a
second CLR type to exist), and SparkplugCodec (Task 16, landed concurrently)
already casts straight to the generated type. A duplicate enum would be the
same enum-drift hazard this repo already names systemic (CLAUDE.md's driver
enum-serialization bug) and would force every downstream task to cast
between two value-compatible-but-nominally-different enums. ToDriverDataType()
maps per design doc SS3.5: Int8/UInt8 widen to Int16/UInt16 (no 8-bit
DriverDataType member), Float/Double to Float32/Float64 (there is no
DriverDataType.Double), Text/UUID/Bytes/File to String, *Array variants to
their element type (IsSparkplugArray carries the array bit separately), and
DataSet/Template/PropertySet/PropertySetList/Unknown return null -- an
explicit "unsupported, caller must skip+warn" rather than a guessed String.
The completeness test enumerates the live generated DataType member set
(via the alias) and asserts every member is mapped or on the explicit
unsupported list, so a future Tahu proto change is caught automatically
instead of silently falling through a stale duplicate enum's default.
Falsifiability verified by hand for three defect shapes (each reverted after
observing RED): wrong Int8 widening, a dropped mapping falling through
undetected, and inverted node/device topic scoping.
Claude-Session: https://claude.ai/code/session_01GASWkNEi68FSCtvr6rLoEW
|
||
|
Joseph Doherty
|
c164ec3da1 |
fix(mqtt): SparkplugCodec's never-throw guard must span the projection too
The try/catch stopped at Payload.Parser.ParseFrom, leaving the metric-projection loop outside it — so the "never throws for any input" contract had a hole in exactly the part that walks an attacker-shaped object graph. Widened to cover parse AND projection. Self-review finding; no test reddened, because a projection escape needs a generated-code shape the vectors do not produce. That is the point: the guard is the contract, not the observed behaviour of today's inputs. Claude-Session: https://claude.ai/code/session_01GASWkNEi68FSCtvr6rLoEW |
||
|
Joseph Doherty
|
2589774480 |
feat(mqtt): SparkplugCodec decode + golden payload vectors
Decodes Sparkplug-B wire bytes into a driver-side projection (SparkplugPayload /
SparkplugMetric / SparkplugValueKind) for the Task 21 ingest state machine.
- Never throws, for ANY input. It sits on MQTTnet's shared dispatcher thread, so
an escaping exception would stall delivery for every subscription on the
connection, not one tag. Garbage, truncation, zero-length, a valid protobuf of
another schema and an over-nested Template all resolve to a verdict.
- Zero-length input is INVALID, not "a payload with no metrics" — protobuf would
parse it as all-defaults, and that is how a truncated-to-nothing body gets
mistaken for a well-formed one.
- Explicit presence throughout (Has{Seq,Name,Alias,Datatype,Timestamp}), never a
zero-check: an NBIRTH legitimately carries seq = 0, and every DATA metric after
a birth carries an alias with no name and no datatype.
- Values are projected RAW, boxed, with the value oneof reported as an explicit
ValueKind — Absent / Null / Scalar / Unsupported all mean different things and
three of them carry a null value. DataSet/Template/extension decode as
Unsupported (v1 scope) rather than throwing or silently vanishing.
- ReinterpretSigned() undoes Sparkplug's two's-complement-in-an-unsigned-field
encoding of Int8/16/32/64. Kept out of decode because a DATA metric carries no
datatype — only the consumer, holding the birth's alias table, knows which
applies. Skipping it publishes 4294967254 for a tag whose value is -42.
- Datatype is carried as the generated Org.Eclipse.Tahu.Protobuf.DataType; the
SparkplugDataType map is Task 17's and is applied downstream (Task 21).
Golden vectors: nbirth.bin (seq=0, named/aliased catalog, a negative Int32, an
is_null metric, a DataSet metric) + ndata.bin (alias-only metrics, no name, no
datatype) are hand-built by SparkplugGoldenPayloads, committed, and pinned by a
drift guard that rebuilds and byte-compares them on every run — plus a test that
they actually reach the output directory, since a missing copy item is invisible
in source.
Claude-Session: https://claude.ai/code/session_01GASWkNEi68FSCtvr6rLoEW
|
||
|
Joseph Doherty
|
4ad540376d |
merge: SQL poll driver (feat/sql-poll-driver)
Read-only Sql Equipment-kind driver: polls SQL Server tables/views on an interval and publishes selected columns/rows as OPC UA variable nodes. Three projects mirror the Modbus split (Contracts / runtime / Browser); ISqlDialect seam (SqlServer only in v1); grouped one-query-per-source reads with a three-layer client-side deadline; schema-walk address picker; typed AdminUI driver-config + tag-config editors; env-gated central-SQL integration + a dedicated-container blackhole gate. Live /run gate PASSED end-to-end on docker-dev: authored a Sql driver + device + KeyValue tag through the AdminUI, deployed, and read the live value (dbo.TagValues.Line1.Speed = 42.5, Good) back over OPC UA. Follow-ups filed: Gitea #496 (§8.1 catalog gate), #497 (cross-driver status codes), #498 (connectionString persist guard). Claude-Session: https://claude.ai/code/session_01GASWkNEi68FSCtvr6rLoEW |
||
|
Joseph Doherty
|
57a5bb3c60 |
chore(docker-dev): wire Sql__ConnectionStrings__DevSql for the Sql driver live gate
Adds the named connection-string ref the Sql poll driver resolves (connectionStringRef: "DevSql") to both central nodes' env, pointing at a DevSql database on the rig's own sql container. Committed-dev-secret exception, same posture as the ConfigDb line above it. This is the env wiring the Task 21 live /run gate used to author + deploy a Sql driver and read a live value (dbo.TagValues.Line1.Speed = 42.5) back over OPC UA. Claude-Session: https://claude.ai/code/session_01GASWkNEi68FSCtvr6rLoEW |
||
|
Joseph Doherty
|
163dd7ab5c |
fix(mqtt): unbreak the .Contracts csproj -- '--' is illegal in an XML comment
The build-platform note added alongside the proto wiring quoted docker-dev's `FROM --platform=linux/amd64` verbatim inside an MSBuild comment. XML forbids '--' in a comment, so MSBuild refused to load the project at all (MSB4025) -- a total build stop for every consumer of .Contracts, i.e. the whole MQTT driver, the Host, and the test suite. Reworded to name the platform in prose. Caught by rebuilding after the edit; the preceding commit was made from a build that predated it. Claude-Session: https://claude.ai/code/session_01GASWkNEi68FSCtvr6rLoEW |
||
|
Joseph Doherty
|
1ae26675ac |
feat(mqtt): vendor Tahu sparkplug_b.proto + Grpc.Tools codegen
Vendors the NORMATIVE (proto2) Eclipse Tahu sparkplug_b.proto and compiles it
in .Contracts with build-time Grpc.Tools, message-only (GrpcServices="None") --
Sparkplug rides MQTT, there is no gRPC service, so no Grpc.Core.Api is pulled in.
Provenance rides in the file header: repo, path, pinned commit
5736e404889d4b95910613040a99ba79589ffb13, permalink, git blob SHA-1
bf72ab5f..., content SHA-256 4432c5c4..., EPL-2.0, and a re-verify recipe. The
body below line 38 is byte-for-byte upstream; the blob SHA-1 matches the tree
entry at that commit, so the copy is provably genuine and not reconstructed.
Chose the proto2 file over upstream's sibling sparkplug_b_c_sharp.proto: the
sibling is a lossy proto3 restatement that drops explicit presence, and protoc
generates valid C# from proto2 into the identical namespace
(Org.Eclipse.Tahu.Protobuf, PascalCased from the package -- no
csharp_namespace option). Presence matters downstream: an NBIRTH legitimately
carries seq = 0 and a DATA metric legitimately omits `name`, so Has{Seq,Name,
Alias,IsNull,Datatype} is the difference between "absent" and "present, zero".
.Contracts stays transport-free: Google.Protobuf is a serialization dependency
with a framework-only graph, Grpc.Tools is PrivateAssets=all, and the resolved
graph is exactly {Google.Protobuf, Grpc.Tools, Core.Abstractions} -- no MQTTnet.
Codegen tests pin the round-trip, the namespace, the Sparkplug field numbers,
the DataType spec indices, and protoc's enum-name mangling (UInt64 -> Uint64,
UUID -> Uuid) that Task 17's datatype map has to spell correctly.
Claude-Session: https://claude.ai/code/session_01GASWkNEi68FSCtvr6rLoEW
|
||
|
Joseph Doherty
|
92ba120964 |
feat(mqtt): MqttDriverForm + MqttDeviceForm — the driver/device config forms P1 omitted
The P1 live gate had to insert MQTT driver config directly via SQL: DriverConfigModal and DeviceModal both fell through to "No typed config form for driver type Mqtt" with no raw-JSON fallback, so broker host/port/TLS/credentials were unauthorable from the AdminUI. Task 12 built the *tag* editor; these are the driver + device surfaces. MqttDriverForm authors the whole MqttDriverOptions connection surface (host/port/clientId, TLS + CA pin, credentials, protocol version/clean-session/keep-alive, connect timeout, reconnect backoffs, mode, maxPayloadBytes, and the Plain sub-object). The Sparkplug sub-object stays P2 — a marked placeholder mirroring MqttTagConfigEditor's stub, with any existing sparkplug keys preserved untouched. The connection is authored on the DRIVER, not the device — unlike Modbus/S7/OpcUaClient and contrary to what docs/drivers/Mqtt.md said. DriverDeviceConfigMerger merges a device's keys up only when the driver has exactly ONE device, so a device-authored broker connection vanishes silently the moment a second device is added. MqttDeviceForm is therefore informational (the GalaxyDeviceForm shape) and round-trips DeviceConfig verbatim; it flags legacy connection keys left on a device by a pre-form deployment, because those still win via merge-up. Serialization goes through the ONE shared MqttJson.Options instance (Task 9's decision), never a fifth per-form copy — pinned by an assertion that an ordinal-only reader CANNOT bind the emitted blob, and by extending the fleet-wide DriverPageJsonConverterTests guard to resolve a form's serializer from an explicit external-instance registry when it has no _jsonOpts field. Dropping the converter from MqttJson.Options reddens 3 tests and leaves the symmetric round-trip green — the Task 9 lesson, reproduced. RawTags is stripped from the emitted blob (the deploy artifact owns it) and unknown top-level keys survive a load->save. Validation mirrors the driver's own [Range] bounds inline, and ToOptions() additionally clamps, so an ignored error still cannot persist a connectTimeoutSeconds:0 driver-brick. A non-blank clientId raises the P1 live-gate warning inline (fixed ids make redundant pair nodes evict each other while both report Healthy). AdminUI 761/761 green (was 730), TWAE-clean; Driver.Mqtt 266/266 green. Claude-Session: https://claude.ai/code/session_01GASWkNEi68FSCtvr6rLoEW |
||
|
Joseph Doherty
|
b295b24419 |
feat(adminui): author + configure the Sql driver in /raw
The read-only Sql driver's runtime factory/probe and tag-side editors were wired, but the AdminUI could not author the DRIVER: the /raw "New driver" type list, the DriverConfigModal switch, and the legacy identity dropdown all omitted Sql, and no SqlDriverForm existed. Live-driving the AdminUI surfaced this. - Add SqlDriverForm.razor over the SqlDriverConfigDto shape: Provider (SqlServer-only in v1), required connectionStringRef (an env-var NAME, not a connection string — label is explicit), optional poll/operation/command timeouts, maxConcurrentGroups, nullIsBad. Enums serialize by NAME via JsonStringEnumConverter; a connection-string literal can never be authored (no such field is collected); rawTags (composer-owned) + allowWrites (inert, read-only v1) are never emitted. - Wire Sql into DriverConfigModal switch, RawDriverTypeDialog type list, and DriverIdentitySection legacy dropdown. - Add SqlDriverFormContractTests: the exact JSON the form emits constructs a driver through the real factory, missing connectionStringRef is rejected, and provider round-trips as the "SqlServer" name (never an ordinal). Claude-Session: https://claude.ai/code/session_01GASWkNEi68FSCtvr6rLoEW |
||
|
Joseph Doherty
|
07a4a5ff5e |
docs(mqtt): P1 plain-MQTT milestone live-verified; endpoint recorded
Task 14 — the P1 milestone gate. Ran the live /run verification on a docker-dev
rig against the real Mosquitto TLS+auth fixture, and recorded the result.
The gate did its job: it found MQTT was unauthorable in production.
RawDriverTypeDialog's driver-type list is a hand-maintained array that nobody
added MQTT to, so the /raw "New driver" picker never offered it — with every
other layer (contracts, driver, browser, factory, host registration, typed tag
editor) complete and 266 green unit tests. Nothing tied that array to
DriverTypeNames and AdminUI has no bUnit, so no offline test could see it.
Fixed, plus RawDriverTypeDialogParityTests: a reflection parity guard that
fails for ANY DriverTypeNames entry missing from the picker. Verified
falsifiable — RED with "…cannot be authored from the /raw New-driver picker,
so they are unreachable in production: Mqtt" before the one-line fix.
A second gap is left OPEN and documented, not fixed (no task ever covered it):
MqttDriverForm / MqttDeviceForm do not exist, and DriverConfigModal/DeviceModal
have no raw-JSON fallback — so an operator can create an MQTT driver but cannot
author its broker endpoint or credentials. P1 is not operator-complete until
those two razor forms land. The gate authored config via SQL to get past it.
Live gate result (isolated 2-node MAIN stack, image built from this branch,
fixture at 10.100.0.35:8883, AllowUntrustedServerCertificate rather than
pinning the CA into the rig):
- typed MQTT tag editor renders and round-trips; Json shows the JSON-path
field, Raw/Scalar hide it; wildcard topic a/+/c blocked at save; data-type
list offers Float64 and no Double; qos/retainSeed absent on "(driver
default)" and qos:2 written as a number; isHistorized/historianTagname
survive a topic-only edit; SparkplugB renders the P2 placeholder and
switching back keeps Plain values; jsonPath is guidance not a gate in all
three shapes ($ seeded only for a brand-new blank tag, blank stays absent,
clearing saves fine)
- deployed, and both editor-written blobs resolved and served changing live
values through OPC UA (22.8 / 1629, StatusCode Good) — no BadNodeIdUnknown
- the bespoke #-observation browser drove a real broker session and rendered
the fixture's actual topic tree (line1/{counter,state,temperature},
noise/chatter, retained/seed); a Modbus device's picker still correctly
reports "Browsing unavailable"
Also found live and documented (not a code change): both nodes of a redundant
pair run the driver, so a FIXED clientId makes them evict each other forever —
the broker logs "already connected, closing old connection" and both nodes
reconnect every ~2s while still reporting Healthy. Unset (the default) is
correct; confirmed 0 reconnects/60s on both nodes after removing it.
Suites: offline 1134 passed / 0 failed (266 Driver.Mqtt + 730 AdminUI incl. the
3 new guards + 138 Core.Abstractions); live 7/7 against the fixture.
Docs: new docs/drivers/Mqtt.md + Mqtt-Test-Fixture.md (the per-driver + fixture
convention every other driver follows), MQTT rows in docs/drivers/README.md,
TestConnectProbes.md, root README.md, and infra/README.md §3.
CLAUDE.md drift corrected while adding the MQTT endpoints:
- the "every fixture carries project: lmxopcua" claim was false — only the
MQTT fixture does; the filter returns one stack, not the fleet
- lmxopcua-fix.ps1 is Windows-VM-only; on macOS drive the host over ssh/rsync
(and the docker-dev rig itself runs locally under OrbStack)
Claude-Session: https://claude.ai/code/session_01GASWkNEi68FSCtvr6rLoEW
|
||
|
Joseph Doherty
|
2409d05a28 |
fix(mqtt): a broker-refused CONNECT no longer reports a Healthy driver
MQTTnet 5 does not throw on an unsuccessful CONNACK — it returns the reason in MqttClientConnectResult.ResultCode and v5 removed v4's ThrowOnNonSuccessfulConnectResponse, so ConnectCoreAsync's unconditional SetState(Connected) sealed a wrong-password deployment green: DriverState.Healthy / HostState.Running on a session that never authenticated. Caught by the Task-13 Mosquitto fixture; 240 offline tests missed it. ConnectCoreAsync now inspects the CONNACK and raises MqttConnectRejectedException. Credentials/identity/protocol/Last-Will refusals are unrecoverable and set Faulted, which stops the reconnect supervisor; transient refusals (ServerUnavailable, ServerBusy, QuotaExceeded, UseAnotherServer, ConnectionRateExceeded, and anything unrecognised) stay Reconnecting under backoff. MqttDriverProbe now shares the rejection wording, so the AdminUI Test-connect button and the running driver can no longer disagree. Claude-Session: https://claude.ai/code/session_01GASWkNEi68FSCtvr6rLoEW |
||
|
Joseph Doherty
|
033b3700c4 |
test(mqtt): Mosquitto TLS+auth fixture + env-gated plain live suite
Adds the MQTT driver integration-test project: an eclipse-mosquitto fixture running auth AND TLS (allow_anonymous false on both listeners), a mosquitto_pub-based JSON publisher emitting retained messages, a cert/password generator script whose output is gitignored, and a live suite gated on MQTT_FIXTURE_ENDPOINT that skips clean offline. The fixture is never anonymous by design: an anonymous broker would let the driver's TLS/CA-pin and auth paths go untested, and those fail silently. The CA-pin leg carries its own falsifiability control (a foreign CA generated in-process must be rejected). Live gate on 10.100.0.35: 7/7 passed. It found a driver defect, reported not fixed here (src/ is out of this task's scope): MqttConnection.ConnectAsync RETURNS NORMALLY when Mosquitto rejects a CONNECT as not-authorized -- MQTTnet 5 does not throw on an unsuccessful CONNACK, so a wrong broker password yields DriverState.Healthy from InitializeAsync. The auth-negative test therefore asserts the invariant that holds either way (no session is ever established) and documents the finding. Claude-Session: https://claude.ai/code/session_01GASWkNEi68FSCtvr6rLoEW |
||
|
Joseph Doherty
|
ff62b62a55 |
refactor(mqtt): accept a blank jsonPath; guide with a seeded "$" instead
Review follow-up. Dropped the hard Validate rejection of a blank jsonPath under a Json payload: the runtime defaults it to the document root, which is the real "publisher puts a bare JSON scalar on the topic" case, so rejecting it made the editor refuse what the driver accepts — inverting "authoring surface accepts <=> publish accepts" — and blocked whole CSV-import batches, since this validator also gates RawManualTagEntryModal's review grid. Replaced with guidance: an UNAUTHORED tag (no topic AND no jsonPath) seeds the field with "$", so a fresh tag emits an explicit "jsonPath":"$" while an existing path-less tag keeps the key ABSENT and the driver defaults it. The wildcard-topic rule stays strict — a wildcard has no legitimate single-Tag interpretation. Also: omit a blank "topic" rather than writing "topic":"", and record why the _lastConfigJson guard diverges from the Modbus template — the landmine is a DERIVED, NON-PERSISTED UI FIELD INFERRED FROM BLOB CONTENT (Mode), which Task 24's Sparkplug field group will be tempted to add more of. Named the host-side dependency (RawTagModal only mutates through this callback) that keeps the staleness trade benign today. Claude-Session: https://claude.ai/code/session_01GASWkNEi68FSCtvr6rLoEW |
||
|
Joseph Doherty
|
a13ae926a8 |
fix(mqtt): gate DisposeAsync against in-flight lifecycle ops (C1) + rebuild-branch tests (I1)
C1 (Critical) — DisposeAsync went straight to TeardownAsync with no lifecycle-gate
wait, reopening the orphaned-connection class. Interleaving: ReinitializeAsync's
rebuild branch holds the gate mid-InitializeCoreAsync, past its own teardown but
before `_connection = connection`; an ungated dispose sees a null connection, does
nothing, and sets `_disposed`; the rebuild then publishes a live connection plus a
host-probe loop that every later dispose short-circuits past. Not proven reachable
today (DriverInstanceActor.PostStop calls the gated ShutdownAsync), but MqttDriver
publicly implements IAsyncDisposable, which invites `await using`.
- DisposeAsync now takes the gate with the same bounded wait + fallback teardown
ShutdownAsync uses, and sets `_disposed` BEFORE the wait.
- InitializeCoreAsync re-checks `_disposed` after the connect and disposes the
connection it just built rather than publishing it — this closes the residual
window on the bounded-timeout fallback path.
- The doc comment no longer claims parity with ShutdownAsync it did not have, and
stops conflating "don't Dispose() the semaphore object" with "don't WaitAsync".
I1 (Important) — the SameSession == false rebuild branch had no test driving it.
Adds three: an endpoint-changing delta rebuilds and Faults on a refused connect;
an ingest-only delta (MaxPayloadBytes) ALSO rebuilds, pinning that IngestIdentity
is nested inside SessionIdentity; and DisposeAsync serializes behind a lifecycle
operation parked at a new internal BeforeConnectHookForTests seam (mirroring
MqttConnection's AfterConnectHookForTests, which exists for the same race class).
Minors: comments recording that IngestIdentity names no Sparkplug field and must
grow one in P2 (tasks 21/22), and why _options/_subscriptions/_authoredRawPaths
get looser memory discipline than _health/_hostState.
Falsifiability: reverting DisposeAsync to the ungated form reddens exactly the new
serialization test ("Shouldly.ShouldAssertException : raced"); forcing SameSession
to always-true reddens exactly the two new rebuild tests. 240/240 MQTT tests pass;
forced rebuild of the driver project is 0 warnings under TreatWarningsAsErrors.
Claude-Session: https://claude.ai/code/session_01GASWkNEi68FSCtvr6rLoEW
|
||
|
Joseph Doherty
|
f79d13e2d8 |
feat(mqtt): factory + DriverTypeNames.Mqtt + host factory/probe registration
Task 9. Adds MqttDriverFactoryExtensions (DriverTypeName = DriverTypeNames.Mqtt, direct deserialization of MqttDriverOptions — no intermediate DTO, mirroring OpcUaClientDriverFactoryExtensions), the DriverTypeNames.Mqtt constant, and both host wiring sites: the factory in DriverFactoryBootstrap.Register and the probe in AddOtOpcUaDriverProbes (the admin-node path Program.cs calls in its hasAdmin block — a probe wired only on driver nodes makes Test-connect silently dead). Carried-forward items: - Converges every MQTT config-parsing seam onto ONE JsonSerializerOptions — MqttJson.Options, in .Contracts alongside MqttDriverOptions. It replaces the probe's internal JsonOpts and the browser's separate private copy; the factory, the probe, MqttDriver.ParseOptions and MqttDriverBrowser now all parse through it. .Contracts is the only assembly all four consumers reference, and the browser's reference to the runtime .Driver project is a documented layering exception scheduled for removal — anchoring the options there would resurrect the duplicate the day it goes away. - Replaces the "Mqtt" literals in MqttDriverProbe, MqttDriverBrowser and MqttDriver with the constant (string value unchanged). - Tightens MqttDriverProbeTests.ProbeAsync_EnumAsName: it asserted only Ok == false + non-empty message, which is also exactly what a JSON-parse failure produces — so it stayed green under the very regression it names. It now asserts the probe got past the parse and reached the network. Falsifiability: deleting JsonStringEnumConverter from MqttJson.Options reddens 9 tests across 4 suites, including the tightened probe test (message becomes "Config JSON is invalid: The JSON value could not be converted to MqttProtocolVersion") — which the pre-fix assertions would have passed. Also references the MQTT driver from Core.Abstractions.Tests so DriverTypeNamesGuardTests' reflective bin scan discovers the new factory and the constant/factory parity check stays honest. Claude-Session: https://claude.ai/code/session_01GASWkNEi68FSCtvr6rLoEW |
||
|
Joseph Doherty
|
36abd871a1 |
feat(mqtt): typed tag editor + validator (plain mode)
Thin typed model over a preserved JsonObject key bag, mirroring the sibling <Driver>TagConfigModel template. Key names and strictness rules mirror MqttTagDefinitionFactory rather than inventing an editor-side schema; enums round-trip as NAMES (the factory reads them strictly by name). No FullName key: under v3 a tag is identified by its RawPath, which the factory keys the definition's Name off — a composed identity key here would be dead weight nothing reads. Mode is a UI-only sub-shape selector inferred from the blob (any Sparkplug descriptor key present) and never serialised — the driver takes Plain vs SparkplugB from its DRIVER config. Only Plain Validate() is implemented; the Sparkplug branch is a stub Task 24 fills, and its keys are preserved untouched. Claude-Session: https://claude.ai/code/session_01GASWkNEi68FSCtvr6rLoEW |
||
|
Joseph Doherty
|
420692b6e5 |
feat(mqtt): MqttDriver shell — lifecycle + authored-only discovery (Once)
Composes MqttConnection + MqttSubscriptionManager + LastValueCache into the IDriver / ITagDiscovery / ISubscribable / IReadable / IHostConnectivityProbe / IRediscoverable capability set. - Discovery replays ONLY the authored raw tags (RediscoverPolicy = Once, SupportsOnlineDiscovery = false) — a chatty broker can never auto-provision. - Composition order is register -> AttachTo -> ConnectAsync; the manager's Reconnected handler is passed through unwrapped so its throw-on-total-failure still tears a deaf session down. - ReinitializeAsync applies a tag-only delta in place and never faults on a bad one; a session-changing delta rebuilds. - ReadAsync serves the last-value cache and degrades per reference (BadWaitingForInitialData), never throwing for the batch. - FlushOptionalCachesAsync is a no-op: the last-value cache IS IReadable. - Adds MqttDriverOptions.RawTags (the authored-tag delivery mechanism every other driver's options DTO already has) and promotes MaxPayloadBytes from a manager ctor knob to an operator-facing key. - Converges on MqttDriverProbe.JsonOpts rather than a second, divergent JsonSerializerOptions. Claude-Session: https://claude.ai/code/session_01GASWkNEi68FSCtvr6rLoEW |
||
|
Joseph Doherty
|
211c6ea6d6 |
feat(mqtt): register MqttDriverBrowser (bespoke-first for Mqtt)
Wires the bespoke MqttDriverBrowser into AdminUI's IDriverBrowser registrations alongside OpcUaClient/Galaxy, overriding the universal DiscoveryDriverBrowser fallback for driverType "Mqtt". Claude-Session: https://claude.ai/code/session_01GASWkNEi68FSCtvr6rLoEW |
||
|
Joseph Doherty
|
a856d61510 |
feat(mqtt): MqttDriverProbe CONNECT handshake
Test-connect probe for the Mqtt driver type: parses MqttDriverOptions
(shared enum-as-name JsonSerializerOptions), opens a bounded MQTT CONNECT
via MqttConnection.BuildClientOptions with a distinct -probe-{guid8}
client-id suffix, and classifies the outcome. Live-probed against
MQTTnet 5.2.0.1603 to confirm the real contract: a broker-rejected
CONNACK is a MqttClientConnectResult.ResultCode, never a thrown
exception, and timeout classification checks the deadline token itself
rather than switching on exception type (which varies unpredictably
between MqttConnectingFailedException and bare OperationCanceledException
for the same frozen-peer shape).
Claude-Session: https://claude.ai/code/session_01GASWkNEi68FSCtvr6rLoEW
|
||
|
Joseph Doherty
|
abf3116bd4 |
fix(mqtt): wildcard tags went dark on reconnect — index by filter, not topic
C1 (CRITICAL). AuthoredTable routed any wildcard-authored tag into `Wildcards`
and never into the concrete topic index. But the two paths that reconstruct
state from a SUBSCRIBED FILTER STRING — OnReconnectedAsync's re-subscribe set
and DegradeTopic — both looked up that concrete-only index, and
`_subscribedTopics` keys on the wildcard PATTERN ("f/+/temp"). So:
- reconnect resolved zero filters for a wildcard tag, hit a silent early
return, and issued NO subscribe. The cache keeps its last Good value, so the
tag never turns Bad — it just stops updating forever behind a connection
reporting healthy. Exactly the silent-death mode MqttConnection's own remarks
warn about, left unguarded for wildcards.
- DegradeTopic missed too, so a SUBACK rejection of a wildcard filter degraded
nothing and left the tag at BadWaitingForInitialData.
Fix: AuthoredTable now carries `ByFilter` (EVERY def, keyed by its authored
topic filter, wildcards included) alongside `ByExactTopic` (concrete only) and
`Wildcards`. Delivery matches an incoming published topic — which never carries
a wildcard — so it keeps using ByExactTopic + the comparer scan. Every path
keyed on a filter string uses ByFilter. The distinction is documented on the
record. The silent early return is now a Warning naming the orphaned topics.
Also in this pass:
- I2: bounded decode on the dispatcher thread. A body over `MaxPayloadBytes`
(default 1 MiB, ctor-settable) is refused BEFORE any GetString/JsonDocument
parse and degrades its own tags. Unbounded decode on the shared dispatcher is
paid by every subscription, not just the offending topic. NOTE: promoting this
to an operator-facing MqttDriverOptions key (+ WithMaximumPacketSize) needs a
.Contracts edit this task is scoped out of; flagged in the XML docs.
- I3: integral-valued reals now coerce to integer tags. JS/Python edge gateways
serialize an integer as 5.0, and TryGetInt32/int.TryParse are syntactic — an
Int32 tag fed by such a gateway silently never received data. Parsed via
decimal so integrality and range are exact across Int64/UInt64. A genuinely
fractional 5.5 is still refused, never rounded.
- I4: the JSON document is parsed ONCE per message and shared across the whole
fan-out (the documented "one document, one JSONPath per signal" shape was
re-parsing per tag). Zero-alloc via a ref struct when no Json tag matches.
- I5: pinned the documented JSON-string-holding-a-number coercion end-to-end.
- Minor: Register now prunes `_subscribedTopics` / `_handleByRawPath` entries for
tags a redeploy dropped, so a deleted topic stops being re-subscribed forever.
Claude-Session: https://claude.ai/code/session_01GASWkNEi68FSCtvr6rLoEW
|
||
|
Joseph Doherty
|
4ec01bdfc1 |
fix(mqtt): bound browse observation against a chatty or malicious broker
Three review findings, all "unbounded work/memory from untrusted broker input" rather than correctness bugs - but this points at a live plant broker. 1. InferPayload decoded the WHOLE payload on MQTTnet's dispatcher thread before trimming to 64 chars. The node cap never engages here: a multi-MB blob on an already-known topic creates no node, it just updates one, so the cost repeats per message forever. Now at most PayloadInspectMaxBytes (512) are examined. A blind slice can cut a multi-byte sequence, which the strict decoder would report as "binary" - TrimToUtf8Boundary backs the window off to the last whole sequence so good UTF-8 is never mislabelled. A clipped payload is String by construction rather than inferred from a partial view. 2. The node cap bounded COUNT, not bytes. MQTT topics run to ~65KB, so 50k nodes near that ceiling is a multi-GB tree. Topics now bounded at 1024 chars and segments at 256, rejected whole (a truncated topic is a DIFFERENT topic the picker would commit as a tag address) and surfaced via an __oversized__ marker kept distinct from __truncated__ - the operator's remedy differs. 3. Control characters in a snippet would break the picker's single-line row; Trim() only strips the ends. Now scrubbed to spaces. Each guard was falsified independently by neutering it and confirming exactly one test reddens. That found a real gap: the oversized-topic test was passing via the SEGMENT bound, leaving the whole-topic bound untested - the test now uses many short segments so only the topic bound can reject it. Also records the .Driver project reference as a known, deliberate exception to the .Browser -> .Contracts pattern, with its cost (AdminUI gains Core + Polly + Serilog) and the clean fix (a leaf transport project; NOT a move into .Contracts, which is deliberately transport-free). Claude-Session: https://claude.ai/code/session_01GASWkNEi68FSCtvr6rLoEW |
||
|
Joseph Doherty
|
d421487bcd |
feat(mqtt): plain subscribe→OnDataChange with retained seed + ref resolver
MqttSubscriptionManager is the P1 plain-MQTT ingest path: it holds the authored RawPath → MqttTagDefinition table behind the shared EquipmentTagRefResolver, establishes one SUBSCRIBE per distinct authored topic, and turns each inbound message into a LastValueCache update plus an OnDataChange notification. Load-bearing choices, each pinned by a falsified test: - The published reference IS the RawPath (def.Name), never the topic and never the TagConfig blob — DriverHostActor's dual-namespace fan-out is RawPath-keyed, so any other key passes every unit test and delivers nothing in production. - One message fans out to EVERY tag on its topic; several tags routinely share a topic with different jsonPaths, and stopping at the first match silently starves the rest. - An unauthored topic raises nothing: MQTT ingest never auto-provisions. - Wildcard-authored topics (accepted by the parser, warned at deploy) match via MQTTnet's own MqttTopicFilterComparer, so '+'/'#' behave as a broker would. Concrete topics stay a single lookup; the wildcard scan runs only when one exists. - Retained seeds are honoured natively on MQTT 5.0 (retain handling) AND filtered client-side on the retain flag, because 3.1.1 brokers always replay. - Nothing throws on MQTTnet's dispatcher thread: a malformed payload degrades that tag (BadDecodingError / BadTypeMismatch), and a throwing OnDataChange subscriber is contained without starving the tags behind it. - The manager issues the FIRST subscribe itself — Reconnected deliberately does not fire after the initial ConnectAsync. - Reconnect re-subscribe: total failure THROWS (MqttConnection then tears the session down and retries under backoff, rather than serving a connected-but-deaf driver); a partial SUBACK rejection degrades only the denied refs, because retrying forever over one ACL-denied topic would take every tag dark. MqttConnection gains a MessageReceived observer (fired on the dispatcher, throwing observers contained) and a bounded SubscribeAsync under its own linked-CTS deadline — deliberately not MqttClientOptions.Timeout, which already governs every MQTTnet op. A refused filter is an outcome, not an exception; only the SUBSCRIBE itself failing throws. Classify() is parameterised by operation name (messages unchanged for connect). JSONPath is a documented subset ($, $.a.b, $['a'], $.a[0]); filters/slices/ recursive descent report a miss rather than pulling in a JSONPath package for an expression form that selects a set where a tag needs one scalar. Claude-Session: https://claude.ai/code/session_01GASWkNEi68FSCtvr6rLoEW |
||
|
Joseph Doherty
|
e27eb77187 |
docs(modbus-rtu): record Wave-1 RTU-over-TCP live /run gate result (PASSED)
Claude-Session: https://claude.ai/code/session_01GASWkNEi68FSCtvr6rLoEW |
||
|
Joseph Doherty
|
f2a9ee5d93 |
docs(mqtt): flag the Last-Will landmine at the browse-options seam
A will is published by the BROKER, so it is invisible to PublishCountForTest. MqttDriverOptions carries none today, but Sparkplug NDEATH is a will message - once P2 adds it, an ungraceful browse disconnect would fire NDEATH under the plant's own edge-node identity. ToBrowseOptions is where it must be cleared. Claude-Session: https://claude.ai/code/session_01GASWkNEi68FSCtvr6rLoEW |
||
|
Joseph Doherty
|
7980b34692 |
feat(mqtt): bespoke passive #-observation browser (plain)
MQTT has no discovery protocol, so the AdminUI address picker learns the topic
space the only way there is: subscribe to the wildcard and accumulate what
arrives. MqttBrowseSession serves that observation as a segment tree (split on
'/'); MqttDriverBrowser opens it with a distinct "{clientId}-browse-{guid8}"
identity under a 5-30 s clamped budget.
Browsing publishes NOTHING - the load-bearing safety property, since the picker
runs against a live plant broker. Every outbound message must route through the
single PublishAsync seam that counts into PublishCountForTest; P2's operator-
triggered Sparkplug rebirth is the one intended exception. Sparkplug mode fails
fast rather than serving a raw spBv1.0 topic tree that looks like a metric tree.
Deviation from the plan's ref list: the project also references .Driver so the
browse CONNECT reuses MqttConnection.BuildClientOptions (TLS posture, CA-pin
chain validator, credentials). A second copy would be a security divergence.
Claude-Session: https://claude.ai/code/session_01GASWkNEi68FSCtvr6rLoEW
|
||
|
Joseph Doherty
|
585f827ef3 |
fix(sql): close the residual credential leak in HandlePollError
The C1 credential-hygiene fix converted InitializeAsync and ReadAsync to
surface only ex.GetType().Name, but left HandlePollError building its
operator-facing Degrade() message from raw ex.Message, gated only by
'if (ex is DbException) return;'. That guard exists for the I1
double-classification concern, not for message safety, and it is incomplete
for the leak vector: a malformed connection string throws ArgumentException
from the keyword parser (the same unquoted-';'-in-password shape the sibling
SqlDriverBrowser.Sanitize special-cases), which is not a DbException and so
reaches Degrade(ex.Message) unredacted.
It is unreachable in today's control flow only because the connection string
is static and validated identically at Initialize first — an emergent
property, not an enforced invariant, and a live per-poll leak the moment a
future edit (refresh-on-reinit, a different provider) breaks that assumption.
Make the fallback type-only like the other two sites; the full exception still
reaches the log sink via the exception parameter. The I1 defer + control tests
are unaffected (they assert Degraded, not message text). Pinned by a new test
driving a credential-bearing ArgumentException through HandlePollError;
verified load-bearing (red against Degrade(ex.Message), green after).
Closes the C1 review finding on commit
|
||
|
Joseph Doherty
|
be1df2d1e5 |
feat(sql): SqlTagConfigEditor razor shell
Claude-Session: https://claude.ai/code/session_01GASWkNEi68FSCtvr6rLoEW |
||
|
Joseph Doherty
|
9b5a96876e |
feat(sql): register Sql factory + probe in Host, add DriverTypeNames.Sql
Add DriverTypeNames.Sql (+ All) as the shared source of truth, and wire the driver into the Host: register the factory in DriverFactoryBootstrap.Register via Driver.Sql.SqlDriverFactoryExtensions.Register(registry, loggerFactory), add the SqlProbe (= Driver.Sql.SqlDriverProbe) probe via TryAddEnumerable, and add the Driver.Sql project reference to the Host. Default tier A (SQL client is managed + cross-platform, so ShouldStub needs no change). Repoint the interim SqlDriver.DriverTypeName / SqlDriverFactoryExtensions.DriverTypeName literals at DriverTypeNames.Sql; both still resolve to "Sql", so the AdminUI TagConfigEditorMap/TagConfigValidator keys and the DriverTypeName-parity test stay green. The Core guard test discovers factories from its own bin, so it also gains a Driver.Sql project reference — that is the "registered-factory side" that lets DriverTypeNamesGuardTests' bidirectional-parity check pass (4/4 green). Claude-Session: https://claude.ai/code/session_01GASWkNEi68FSCtvr6rLoEW |
||
|
Joseph Doherty
|
fcc7ed698e |
harden(modbus-rtu): factory throws on unknown transport + document RTU FC-shape assumption
Claude-Session: https://claude.ai/code/session_01GASWkNEi68FSCtvr6rLoEW |
||
|
Joseph Doherty
|
a6370a26f8 |
fix(mqtt): make connect idempotent, bound the reconnect callback, stop State lying
Closes the connect-vs-connect defect: MQTTnet throws (and raises no DisconnectedAsync) on a connect-while-connected, so a caller's ConnectAsync and the reconnect supervisor corrupted each other in both directions. Both paths now check first; when the supervisor finds the session already restored it stands down WITHOUT firing Reconnected. Reconnected becomes Func<CancellationToken, Task>, fed from the lifetime token and capped at ConnectTimeoutSeconds, so a hung re-subscribe can no longer park the supervisor. Connected is published only after the re-subscribe succeeds; a supervisor that dies now reports Faulted instead of Reconnecting forever. Claude-Session: https://claude.ai/code/session_01GASWkNEi68FSCtvr6rLoEW |
||
|
Joseph Doherty
|
898c7365c4 |
test(sql): close the blackhole gate's leaked-pause race + bound the shell command
Review follow-up on the blackhole gate. Two robustness gaps, both bounded to the disposable dedicated container (never shared infra), fixed: - If the outer token fired while 'docker pause' was in flight, 'paused' was set only AFTER the await, so a cancellation there left paused=false while the container could still complete the pause — the finally then skipped unpause and leaked a frozen container. Mark paused BEFORE the await, and make the finally's unpause best-effort (unpausing a never-actually-paused container errors harmlessly, and a cleanup failure must never mask the real test outcome). - The pause/unpause shell commands had no wall-clock bound of their own — only the post-pause read was capped — so a hung SSH handshake would hang CI. Give RunShellCommandAsync its own 30s hard cap that kills the process and throws TimeoutException, distinct from an operator's own cancellation. Offline skip still clean (1 skipped, 9ms, no socket/docker). Claude-Session: https://claude.ai/code/session_01GASWkNEi68FSCtvr6rLoEW |